[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:01 dist-test-slave-1dkt krb5kdc[8427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:01 dist-test-slave-1dkt krb5kdc[8427](info): set up 2 sockets
Sep 29 02:52:01 dist-test-slave-1dkt krb5kdc[8427](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 02:52:02.456342 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.002s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:02 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:02.608089 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40319
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:02.918668 8440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:02.918949 8440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:02.919086 8440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:02.926357 8440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:02.926470 8440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:02.926527 8440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:02.926604 8440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:02.926658 8440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:02.936621 8440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:02.937948 8440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:02.939280 8440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:02.951220 8448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:02.951339 8445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:02.951167 8446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:02.951838 8440 server_base.cc:1034] running on GCE node
I20260929 02:52:02.952649 8440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:02.954106 8440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:02.955314 8440 hybrid_clock.cc:648] HybridClock initialized: now 1790650322955268 us; error 34 us; skew 500 ppm
I20260929 02:52:02.955641 8440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:02 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650322, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:02.958530 8440 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:02.960114 8440 webserver.cc:460] Webserver started at http://127.8.57.62:46101/ using document root <none> and password file <none>
I20260929 02:52:02.960762 8440 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:02.960855 8440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:02.961112 8440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:02.964075 8440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "addf97335f4345f5a7de3722e3699ac2"
format_stamp: "Formatted at 2026-09-29 02:52:02 on dist-test-slave-1dkt"
server_key: "11c30bbad7300ad9892adf4afa109e65"
server_key_iv: "e24434207a770e311d427929be7e2bf2"
server_key_version: "encryptionkey@0"
I20260929 02:52:02.964778 8440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "addf97335f4345f5a7de3722e3699ac2"
format_stamp: "Formatted at 2026-09-29 02:52:02 on dist-test-slave-1dkt"
server_key: "11c30bbad7300ad9892adf4afa109e65"
server_key_iv: "e24434207a770e311d427929be7e2bf2"
server_key_version: "encryptionkey@0"
I20260929 02:52:02.969821 8440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:02.973145 8455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:02.974411 8440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:02.974542 8440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "addf97335f4345f5a7de3722e3699ac2"
format_stamp: "Formatted at 2026-09-29 02:52:02 on dist-test-slave-1dkt"
server_key: "11c30bbad7300ad9892adf4afa109e65"
server_key_iv: "e24434207a770e311d427929be7e2bf2"
server_key_version: "encryptionkey@0"
I20260929 02:52:02.974695 8440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:02.992597 8440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:02.996444 8440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:02.996671 8440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:03.013347 8440 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40319
I20260929 02:52:03.013370 8506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40319 every 8 connection(s)
I20260929 02:52:03.014869 8440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:03.018993 8507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:03.022537 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 8440
I20260929 02:52:03.022843 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:03.025301 8420 external_mini_cluster.cc:1428] Setting key 3be92190fd1a20f3a300f560d03ab44f
I20260929 02:52:03.031584 8507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2: Bootstrap starting.
I20260929 02:52:03.034598 8507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:03.035586 8507 log.cc:824] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:03.038334 8507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2: No bootstrap required, opened a new log
I20260929 02:52:03.048149 8507 raft_consensus.cc:348] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "addf97335f4345f5a7de3722e3699ac2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40319 } }
I20260929 02:52:03.048589 8507 raft_consensus.cc:374] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:03.048698 8507 raft_consensus.cc:729] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: addf97335f4345f5a7de3722e3699ac2, State: Initialized, Role: FOLLOWER
I20260929 02:52:03.049409 8507 consensus_queue.cc:260] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "addf97335f4345f5a7de3722e3699ac2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40319 } }
I20260929 02:52:03.049693 8507 raft_consensus.cc:388] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:03.049851 8507 raft_consensus.cc:482] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:03.050016 8507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:03.052362 8507 raft_consensus.cc:504] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "addf97335f4345f5a7de3722e3699ac2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40319 } }
I20260929 02:52:03.052865 8507 leader_election.cc:302] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: addf97335f4345f5a7de3722e3699ac2; no voters:
I20260929 02:52:03.053418 8507 leader_election.cc:288] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:03.053586 8512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:03.054163 8512 raft_consensus.cc:686] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [term 1 LEADER]: Becoming Leader. State: Replica: addf97335f4345f5a7de3722e3699ac2, State: Running, Role: LEADER
Sep 29 02:52:03 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:03.054762 8512 consensus_queue.cc:237] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "addf97335f4345f5a7de3722e3699ac2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40319 } }
I20260929 02:52:03.056037 8507 sys_catalog.cc:564] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:03.057542 8513 sys_catalog.cc:455] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader addf97335f4345f5a7de3722e3699ac2. Latest consensus state: current_term: 1 leader_uuid: "addf97335f4345f5a7de3722e3699ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "addf97335f4345f5a7de3722e3699ac2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40319 } } }
I20260929 02:52:03.057909 8513 sys_catalog.cc:458] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:03.058426 8514 sys_catalog.cc:455] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "addf97335f4345f5a7de3722e3699ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "addf97335f4345f5a7de3722e3699ac2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40319 } } }
I20260929 02:52:03.058700 8514 sys_catalog.cc:458] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:03.061802 8517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:03.067296 8517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:03.068099 8509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:03.029899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.8.57.62:40319)
0929 02:52:03.030779 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:03.030794 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:03.032082 (+ 1288us) server_negotiation.cc:407] Connection header received
0929 02:52:03.040788 (+ 8706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:03.040817 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:03.041339 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:03.041799 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:03.044525 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:03.046634 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:03.048429 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:03.049455 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:03.049642 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:03.055977 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:03.055990 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:03.056011 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:03.056064 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:03.060477 (+ 4413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:03.061731 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:03.061740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:03.061752 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:03.062043 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:03.062734 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:03.062739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:03.062742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:03.063500 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:03.063809 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:03.066293 (+ 2484us) server_negotiation.cc:299] Negotiation successful
0929 02:52:03.066715 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":382,"threads_started":1}
I20260929 02:52:03.077708 8517 catalog_manager.cc:1347] Generated new cluster ID: fcc5a14910ba4f2086886076a129805c
I20260929 02:52:03.077836 8517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:03.092445 8517 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:03.093963 8517 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:03.104542 8517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P addf97335f4345f5a7de3722e3699ac2: Generated new TSK 0
I20260929 02:52:03.105389 8517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:03.155457 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40319
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:03.494377 8533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:03.494674 8533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:03.494822 8533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:03.502035 8533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:03.502167 8533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:03.502326 8533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:03.512174 8533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40319
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:03.513479 8533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:03.514870 8533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:03.527592 8538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:03.527529 8541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:03.527565 8539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:03.528322 8533 server_base.cc:1034] running on GCE node
I20260929 02:52:03.528854 8533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:03.529788 8533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:03.530972 8533 hybrid_clock.cc:648] HybridClock initialized: now 1790650323530922 us; error 55 us; skew 500 ppm
I20260929 02:52:03.531278 8533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:03 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650323, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:03.534197 8533 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:03.535568 8533 webserver.cc:460] Webserver started at http://127.8.57.1:39383/ using document root <none> and password file <none>
I20260929 02:52:03.536263 8533 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:03.536371 8533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:03.536646 8533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:03.539443 8533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "ce1ecbf4a6d84f1a95d94571f642742f"
format_stamp: "Formatted at 2026-09-29 02:52:03 on dist-test-slave-1dkt"
server_key: "8eebdbf434a4dcca7680349a3ff2e283"
server_key_iv: "da24e76f1ddba1d8da0f82158e292956"
server_key_version: "encryptionkey@0"
I20260929 02:52:03.540200 8533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "ce1ecbf4a6d84f1a95d94571f642742f"
format_stamp: "Formatted at 2026-09-29 02:52:03 on dist-test-slave-1dkt"
server_key: "8eebdbf434a4dcca7680349a3ff2e283"
server_key_iv: "da24e76f1ddba1d8da0f82158e292956"
server_key_version: "encryptionkey@0"
I20260929 02:52:03.544962 8533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 02:52:03.548156 8548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:03.549384 8533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:52:03.549525 8533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "ce1ecbf4a6d84f1a95d94571f642742f"
format_stamp: "Formatted at 2026-09-29 02:52:03 on dist-test-slave-1dkt"
server_key: "8eebdbf434a4dcca7680349a3ff2e283"
server_key_iv: "da24e76f1ddba1d8da0f82158e292956"
server_key_version: "encryptionkey@0"
I20260929 02:52:03.549696 8533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:03.575379 8533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:03.579572 8533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:03.579838 8533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:03.580832 8533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:03.582218 8533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:03.582293 8533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:03.582407 8533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:03.582456 8533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:03.611680 8533 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:39299
I20260929 02:52:03.611725 8660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:39299 every 8 connection(s)
I20260929 02:52:03.612926 8533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:03.622545 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 8533
I20260929 02:52:03.622746 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:03.623327 8420 external_mini_cluster.cc:1428] Setting key a4c1f1de1e8ef6e05caa1eb015d8c8a9
Sep 29 02:52:03 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650323, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:03.632896 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:03.615265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:36375 (local address 127.8.57.62:40319)
0929 02:52:03.615719 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:03.615725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:03.616660 (+ 935us) server_negotiation.cc:407] Connection header received
0929 02:52:03.618392 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:03.618397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:03.618466 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:03.618560 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:03.620357 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:03.621091 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:03.622603 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:03.623100 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:03.623218 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:03.626976 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:03.626981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:03.626984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:03.627017 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:03.629914 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:03.630626 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:03.630632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:03.630636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:03.630757 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:03.631162 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:03.631167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:03.631170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:03.631432 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:03.631582 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:03.632319 (+ 737us) server_negotiation.cc:299] Negotiation successful
0929 02:52:03.632557 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":215,"threads_started":1}
I20260929 02:52:03.633806 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:03.615800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40319 (local address 127.8.57.1:36375)
0929 02:52:03.616498 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:03.616546 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:03.618156 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:03.618744 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:03.618752 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:03.619336 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:03.620183 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:03.620209 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:03.621229 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:03.621233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:03.622441 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:03.622457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:03.623320 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:03.623327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:03.623577 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:03.624201 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:03.624223 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:03.626761 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:03.630055 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:03.630064 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:03.630083 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:03.630504 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:03.630853 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:03.630858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:03.630862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:03.631050 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:03.631551 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:03.631560 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:03.631928 (+ 368us) client_negotiation.cc:769] Sending connection context
0929 02:52:03.632180 (+ 252us) client_negotiation.cc:240] Negotiation successful
0929 02:52:03.632700 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":254,"threads_started":1}
I20260929 02:52:03.635294 8661 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40319
I20260929 02:52:03.635584 8661 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:03.636327 8661 heartbeater.cc:511] Master 127.8.57.62:40319 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:03.638417 8472 ts_manager.cc:194] Registered new tserver with Master: ce1ecbf4a6d84f1a95d94571f642742f (127.8.57.1:39299)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:52:03.639994 8472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:36375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:03.658444 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40319
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:03.978335 8667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:03.978747 8667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:03.978914 8667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:03.988984 8667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:03.989117 8667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:03.989279 8667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:03.999176 8667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40319
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:04.000510 8667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:04.001802 8667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:04.014611 8673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:04.015085 8675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:04.015858 8667 server_base.cc:1034] running on GCE node
W20260929 02:52:04.019690 8672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:04.020587 8667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:04.021647 8667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:04.022853 8667 hybrid_clock.cc:648] HybridClock initialized: now 1790650324022790 us; error 63 us; skew 500 ppm
I20260929 02:52:04.023164 8667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:04.026257 8667 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:04.027776 8667 webserver.cc:460] Webserver started at http://127.8.57.2:34265/ using document root <none> and password file <none>
I20260929 02:52:04.028538 8667 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:04.028653 8667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:04.028942 8667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:04.031837 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "de5d5b487899482e9022d660d1e6b42f"
format_stamp: "Formatted at 2026-09-29 02:52:04 on dist-test-slave-1dkt"
server_key: "622d0905ce441ba9e7e41f2e39128f7c"
server_key_iv: "c05ff1d34a0bf78e49deb40ad4fca53c"
server_key_version: "encryptionkey@0"
I20260929 02:52:04.032619 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "de5d5b487899482e9022d660d1e6b42f"
format_stamp: "Formatted at 2026-09-29 02:52:04 on dist-test-slave-1dkt"
server_key: "622d0905ce441ba9e7e41f2e39128f7c"
server_key_iv: "c05ff1d34a0bf78e49deb40ad4fca53c"
server_key_version: "encryptionkey@0"
I20260929 02:52:04.037634 8667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:52:04.041067 8682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:04.042394 8667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:04.042546 8667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "de5d5b487899482e9022d660d1e6b42f"
format_stamp: "Formatted at 2026-09-29 02:52:04 on dist-test-slave-1dkt"
server_key: "622d0905ce441ba9e7e41f2e39128f7c"
server_key_iv: "c05ff1d34a0bf78e49deb40ad4fca53c"
server_key_version: "encryptionkey@0"
I20260929 02:52:04.042737 8667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:04.059945 8667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:04.063954 8667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:04.064227 8667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:04.065171 8667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:04.066504 8667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:04.066583 8667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:04.066705 8667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:04.066757 8667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:04.096015 8667 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:42281
I20260929 02:52:04.096046 8794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:42281 every 8 connection(s)
I20260929 02:52:04.097222 8667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:04.102640 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 8667
I20260929 02:52:04.102819 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:04.103334 8420 external_mini_cluster.cc:1428] Setting key 4807232fe46e3183cdce35041338a556
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:04.118073 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.099616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:34339 (local address 127.8.57.62:40319)
0929 02:52:04.099782 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.099787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.100846 (+ 1059us) server_negotiation.cc:407] Connection header received
0929 02:52:04.102660 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.102664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.102739 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.102827 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.104731 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.105548 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.107713 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.108137 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.108223 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.111820 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.111827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.111832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.111900 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.114975 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.115675 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.115680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.115684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.115836 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.116254 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.116259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.116262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.116626 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.116766 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.117491 (+ 725us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.117728 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:52:04.118788 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.100046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40319 (local address 127.8.57.2:34339)
0929 02:52:04.100702 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:04.100742 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:04.102352 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:04.103146 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:04.103157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:04.103855 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:04.104562 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:04.104581 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.105698 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.105703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:04.107588 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:04.107603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.108338 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.108344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:04.108597 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.109241 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:04.109272 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:04.111596 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:04.115093 (+ 3497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:04.115100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:04.115113 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:04.115564 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:04.115968 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:04.115971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:04.115973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:04.116155 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:04.116753 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:04.116759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:04.117141 (+ 382us) client_negotiation.cc:769] Sending connection context
0929 02:52:04.117366 (+ 225us) client_negotiation.cc:240] Negotiation successful
0929 02:52:04.117733 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":252,"threads_started":1}
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:04.120123 8795 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40319
I20260929 02:52:04.120393 8795 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:04.121136 8795 heartbeater.cc:511] Master 127.8.57.62:40319 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
I20260929 02:52:04.122627 8472 ts_manager.cc:194] Registered new tserver with Master: de5d5b487899482e9022d660d1e6b42f (127.8.57.2:42281)
I20260929 02:52:04.123458 8472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:34339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:04.139806 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40319
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:04.449203 8800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:04.449486 8800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:04.449611 8800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:04.457062 8800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:04.457172 8800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:04.457319 8800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:04.467237 8800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40319
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:04.468526 8800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:04.469834 8800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:04.482682 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:04.482779 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:04.482801 8808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:04.483263 8800 server_base.cc:1034] running on GCE node
I20260929 02:52:04.483966 8800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:04.484923 8800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:04.486140 8800 hybrid_clock.cc:648] HybridClock initialized: now 1790650324486081 us; error 51 us; skew 500 ppm
I20260929 02:52:04.486444 8800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:04.489526 8800 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:04.491050 8800 webserver.cc:460] Webserver started at http://127.8.57.3:37467/ using document root <none> and password file <none>
I20260929 02:52:04.491727 8800 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:04.491842 8800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:04.492161 8800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:04.495076 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "d2b60c193d7e4bec8cfc0bb8fcad3f2d"
format_stamp: "Formatted at 2026-09-29 02:52:04 on dist-test-slave-1dkt"
server_key: "869f29a9bd1f0252a7c4f3e34c443f6a"
server_key_iv: "57200d7774f5b461376bb0284c105b18"
server_key_version: "encryptionkey@0"
I20260929 02:52:04.495814 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "d2b60c193d7e4bec8cfc0bb8fcad3f2d"
format_stamp: "Formatted at 2026-09-29 02:52:04 on dist-test-slave-1dkt"
server_key: "869f29a9bd1f0252a7c4f3e34c443f6a"
server_key_iv: "57200d7774f5b461376bb0284c105b18"
server_key_version: "encryptionkey@0"
I20260929 02:52:04.501235 8800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:52:04.504763 8815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:04.506178 8800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:04.506335 8800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "d2b60c193d7e4bec8cfc0bb8fcad3f2d"
format_stamp: "Formatted at 2026-09-29 02:52:04 on dist-test-slave-1dkt"
server_key: "869f29a9bd1f0252a7c4f3e34c443f6a"
server_key_iv: "57200d7774f5b461376bb0284c105b18"
server_key_version: "encryptionkey@0"
I20260929 02:52:04.506536 8800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:04.539009 8800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:04.543365 8800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:04.543635 8800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:04.544636 8800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:04.546003 8800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:04.546087 8800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:04.546248 8800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:04.546300 8800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:04.575766 8800 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:36595
I20260929 02:52:04.575817 8927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:36595 every 8 connection(s)
I20260929 02:52:04.577029 8800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:04.584705 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 8800
I20260929 02:52:04.584880 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:04.585261 8420 external_mini_cluster.cc:1428] Setting key acb50383973528788deed9c9666e1540
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:04.595607 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.579523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:37953 (local address 127.8.57.62:40319)
0929 02:52:04.579697 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.579701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.580864 (+ 1163us) server_negotiation.cc:407] Connection header received
0929 02:52:04.582579 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.582583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.582650 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.582734 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.584283 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.585118 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.586747 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.587139 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.587250 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.590547 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.590553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.590556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.590589 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.592815 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.593475 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.593480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.593484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.593604 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.593991 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.593995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.593997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.594259 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.594385 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.595067 (+ 682us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.595279 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:52:04.596577 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.580004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40319 (local address 127.8.57.3:37953)
0929 02:52:04.580700 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:04.580742 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:04.582320 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:04.582927 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:04.582935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:04.583469 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:04.584103 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:04.584123 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.585255 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.585259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:04.586626 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:04.586637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.587348 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.587355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:04.587595 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.588277 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:04.588304 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:04.590333 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:04.592941 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:04.592950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:04.592968 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:04.593358 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:04.593697 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:04.593702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:04.593706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:04.593879 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:04.594373 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:04.594380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:04.594699 (+ 319us) client_negotiation.cc:769] Sending connection context
0929 02:52:04.594929 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 02:52:04.595438 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
I20260929 02:52:04.597891 8928 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40319
I20260929 02:52:04.598165 8928 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:04.598862 8928 heartbeater.cc:511] Master 127.8.57.62:40319 requested a full tablet report, sending...
I20260929 02:52:04.600342 8472 ts_manager.cc:194] Registered new tserver with Master: d2b60c193d7e4bec8cfc0bb8fcad3f2d (127.8.57.3:36595)
I20260929 02:52:04.601084 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:52:04.601289 8472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:37953
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:04.641413 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.627064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.8.57.1:39299)
0929 02:52:04.627670 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.627679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.627709 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:52:04.627830 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.627837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.628093 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.628231 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.629594 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.630594 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.631816 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.632418 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.632541 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.636172 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.636182 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.636196 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.636228 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.638707 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.639332 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.639340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.639342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.639465 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.639848 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.639852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.639854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.640233 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.640512 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.640802 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.641075 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":290,"threads_started":1}
W20260929 02:52:04.642764 8576 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55720
I20260929 02:52:04.642876 8661 heartbeater.cc:503] Master 127.8.57.62:40319 was elected leader, sending a full tablet report...
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:04.681841 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.668689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55734 (local address 127.8.57.1:39299)
0929 02:52:04.668911 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.668917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.669174 (+ 257us) server_negotiation.cc:407] Connection header received
0929 02:52:04.669332 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.669336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.669427 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.669502 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.670756 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.671470 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.672692 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.673080 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.673166 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.676832 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.676838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.676843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.676887 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.679533 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.680225 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.680230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.680232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.680381 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.680697 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.680700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.680702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.680949 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.681120 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.681370 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.681536 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:52:04.683233 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 8533
I20260929 02:52:04.694851 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.684156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.8.57.1:39299)
0929 02:52:04.684306 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.684311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.684733 (+ 422us) server_negotiation.cc:407] Connection header received
0929 02:52:04.684837 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.684841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.684895 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.684966 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.686002 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.686674 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.687934 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.688382 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.688472 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.690486 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.690491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.690494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.690521 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.692869 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.693446 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.693450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.693452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.693577 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.693829 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.693833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.693836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.694072 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.694249 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.694442 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.694593 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:52:04.695494 8656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:04.777199 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 8533
I20260929 02:52:04.789189 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 8667
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:04.804113 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.789999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.8.57.2:42281)
0929 02:52:04.790483 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.790490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.790514 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:52:04.790588 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.790594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.790793 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.790946 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.792119 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.793140 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.794346 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.794883 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.794994 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.798748 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.798759 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.798773 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.798807 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.801454 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.802087 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.802093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.802096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.802221 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.802577 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.802580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.802582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.802919 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.803197 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.803486 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.803746 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":244,"threads_started":1}
I20260929 02:52:04.805325 8790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:04.887104 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 8667
I20260929 02:52:04.898633 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 8800
Sep 29 02:52:04 dist-test-slave-1dkt krb5kdc[8427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:04.913661 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:04.899559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.8.57.3:36595)
0929 02:52:04.900233 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:04.900241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:04.900269 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:52:04.900329 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:04.900335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:04.900552 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:04.900679 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:04.901777 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.902811 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.904065 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:04.904645 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:04.904749 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:04.908521 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:04.908533 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:04.908548 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:04.908584 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:04.911236 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.911800 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.911807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.911809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.911963 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:04.912310 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:04.912314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:04.912316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:04.912673 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:04.912916 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:04.913105 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 02:52:04.913302 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":366,"threads_started":1}
I20260929 02:52:04.914832 8923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:04.996598 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 8800
I20260929 02:52:05.009037 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 8440
I20260929 02:52:05.010202 8502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:05.112087 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 8440
2026-09-29T02:52:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3644 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:05 dist-test-slave-1dkt krb5kdc[8949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:05 dist-test-slave-1dkt krb5kdc[8949](info): set up 2 sockets
Sep 29 02:52:05 dist-test-slave-1dkt krb5kdc[8949](info): commencing operation
krb5kdc: starting...
W20260929 02:52:06.107741 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.005s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:06 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:06.271167 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:38399
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:38399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:06.581435 8962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:06.581725 8962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:06.581839 8962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:06.589128 8962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:06.589236 8962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:06.589305 8962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:06.589382 8962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:06.589442 8962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:06.599114 8962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:38399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:38399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:06.600392 8962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:06.601635 8962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:06.612690 8970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:06.612723 8968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:06.612747 8967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:06.613541 8962 server_base.cc:1034] running on GCE node
I20260929 02:52:06.614269 8962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:06.615682 8962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:06.616880 8962 hybrid_clock.cc:648] HybridClock initialized: now 1790650326616811 us; error 58 us; skew 500 ppm
I20260929 02:52:06.617190 8962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:06 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:06.624805 8962 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:06.626362 8962 webserver.cc:460] Webserver started at http://127.8.57.62:33339/ using document root <none> and password file <none>
I20260929 02:52:06.627080 8962 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:06.627187 8962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:06.627452 8962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:06.630502 8962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "866b1624727e48aa8322e094347c51c1"
format_stamp: "Formatted at 2026-09-29 02:52:06 on dist-test-slave-1dkt"
server_key: "d4bed0b4634faf13779442a8fa8b1dbb"
server_key_iv: "ca8ebbf2d2e045d3694dc0a85db67c03"
server_key_version: "encryptionkey@0"
I20260929 02:52:06.631238 8962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "866b1624727e48aa8322e094347c51c1"
format_stamp: "Formatted at 2026-09-29 02:52:06 on dist-test-slave-1dkt"
server_key: "d4bed0b4634faf13779442a8fa8b1dbb"
server_key_iv: "ca8ebbf2d2e045d3694dc0a85db67c03"
server_key_version: "encryptionkey@0"
I20260929 02:52:06.636497 8962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:52:06.640098 8977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:06.641506 8962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:06.641669 8962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "866b1624727e48aa8322e094347c51c1"
format_stamp: "Formatted at 2026-09-29 02:52:06 on dist-test-slave-1dkt"
server_key: "d4bed0b4634faf13779442a8fa8b1dbb"
server_key_iv: "ca8ebbf2d2e045d3694dc0a85db67c03"
server_key_version: "encryptionkey@0"
I20260929 02:52:06.641834 8962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:06.667349 8962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:06.671365 8962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:06.671605 8962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:06.688687 8962 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:38399
I20260929 02:52:06.688709 9028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:38399 every 8 connection(s)
I20260929 02:52:06.690066 8962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:06.693670 9029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:06.696890 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 8962
I20260929 02:52:06.697103 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:06.697892 8420 external_mini_cluster.cc:1428] Setting key fe94fa9e496585395dbe6882d0a13791
I20260929 02:52:06.705886 9029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1: Bootstrap starting.
I20260929 02:52:06.709131 9029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:06.710114 9029 log.cc:824] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:52:06 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:06.712965 9029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1: No bootstrap required, opened a new log
I20260929 02:52:06.719091 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:06.699975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.8.57.62:38399)
0929 02:52:06.700701 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:06.700712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:06.700753 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:52:06.702367 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:06.702391 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:06.702816 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:06.703194 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:06.704675 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:06.706049 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:06.707268 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:06.708010 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:06.708175 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:06.712037 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:06.712045 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:06.712060 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:06.712094 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:06.715104 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:06.715768 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:06.715775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:06.715781 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:06.715999 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:06.716359 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:06.716363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:06.716367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:06.716883 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:06.717177 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:06.717649 (+ 472us) server_negotiation.cc:299] Negotiation successful
0929 02:52:06.718061 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":325,"threads_started":1}
I20260929 02:52:06.721058 9029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "866b1624727e48aa8322e094347c51c1" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 38399 } }
I20260929 02:52:06.721421 9029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:06.721515 9029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 866b1624727e48aa8322e094347c51c1, State: Initialized, Role: FOLLOWER
I20260929 02:52:06.722126 9029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "866b1624727e48aa8322e094347c51c1" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 38399 } }
I20260929 02:52:06.722388 9029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:06.722508 9029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:06.722659 9029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:06.724803 9029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "866b1624727e48aa8322e094347c51c1" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 38399 } }
I20260929 02:52:06.725337 9029 leader_election.cc:302] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 866b1624727e48aa8322e094347c51c1; no voters:
I20260929 02:52:06.725878 9029 leader_election.cc:288] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:06.725983 9034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:06.726598 9034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [term 1 LEADER]: Becoming Leader. State: Replica: 866b1624727e48aa8322e094347c51c1, State: Running, Role: LEADER
I20260929 02:52:06.727185 9034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "866b1624727e48aa8322e094347c51c1" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 38399 } }
I20260929 02:52:06.727804 9029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:06.729729 9035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "866b1624727e48aa8322e094347c51c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "866b1624727e48aa8322e094347c51c1" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 38399 } } }
I20260929 02:52:06.730084 9035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:06.730861 9036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 866b1624727e48aa8322e094347c51c1. Latest consensus state: current_term: 1 leader_uuid: "866b1624727e48aa8322e094347c51c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "866b1624727e48aa8322e094347c51c1" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 38399 } } }
I20260929 02:52:06.731196 9036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:06.732545 9040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:06.737648 9040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:06.747745 9040 catalog_manager.cc:1347] Generated new cluster ID: 4e0f01b4d4104e16b0dc4b17f40d5b1c
I20260929 02:52:06.747938 9040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:06.758739 9040 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:06.760082 9040 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:06.768987 9040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 866b1624727e48aa8322e094347c51c1: Generated new TSK 0
I20260929 02:52:06.769834 9040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:06.807825 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:38399
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:07.118250 9055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:07.118523 9055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:07.118624 9055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:07.126258 9055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:07.126371 9055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:07.126494 9055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:07.137145 9055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:38399
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:07.138878 9055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:07.140535 9055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:07.153311 9060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:07.153339 9061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:07.153381 9063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:07.154057 9055 server_base.cc:1034] running on GCE node
I20260929 02:52:07.154656 9055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:07.155581 9055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:07.156766 9055 hybrid_clock.cc:648] HybridClock initialized: now 1790650327156717 us; error 43 us; skew 500 ppm
I20260929 02:52:07.157063 9055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:07 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650327, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:07.159936 9055 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:07.161247 9055 webserver.cc:460] Webserver started at http://127.8.57.1:40043/ using document root <none> and password file <none>
I20260929 02:52:07.161913 9055 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:07.162024 9055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:07.162293 9055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:07.165149 9055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "c18ec7d8741b4794b9c393bd44a36dd5"
format_stamp: "Formatted at 2026-09-29 02:52:07 on dist-test-slave-1dkt"
server_key: "a5e251df269de9c5c0fffe9e0dac66b7"
server_key_iv: "148a4eb2644faf4bc677316a1b59da62"
server_key_version: "encryptionkey@0"
I20260929 02:52:07.165910 9055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "c18ec7d8741b4794b9c393bd44a36dd5"
format_stamp: "Formatted at 2026-09-29 02:52:07 on dist-test-slave-1dkt"
server_key: "a5e251df269de9c5c0fffe9e0dac66b7"
server_key_iv: "148a4eb2644faf4bc677316a1b59da62"
server_key_version: "encryptionkey@0"
I20260929 02:52:07.170869 9055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:52:07.174229 9070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:07.175539 9055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:52:07.175684 9055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "c18ec7d8741b4794b9c393bd44a36dd5"
format_stamp: "Formatted at 2026-09-29 02:52:07 on dist-test-slave-1dkt"
server_key: "a5e251df269de9c5c0fffe9e0dac66b7"
server_key_iv: "148a4eb2644faf4bc677316a1b59da62"
server_key_version: "encryptionkey@0"
I20260929 02:52:07.175860 9055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:07.191661 9055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:07.195802 9055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:07.196117 9055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:07.197057 9055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:07.198360 9055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:07.198439 9055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:07.198558 9055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:07.198609 9055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:07.228027 9055 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:40843
I20260929 02:52:07.228096 9182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:40843 every 8 connection(s)
I20260929 02:52:07.229378 9055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:07.233052 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9055
I20260929 02:52:07.233259 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:07.233930 8420 external_mini_cluster.cc:1428] Setting key 8fc87bf50cb7c3efead5d4b427864c9d
Sep 29 02:52:07 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650327, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:07.249572 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:07.231774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:49153 (local address 127.8.57.62:38399)
0929 02:52:07.232207 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:07.232212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:07.233159 (+ 947us) server_negotiation.cc:407] Connection header received
0929 02:52:07.235102 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:07.235107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:07.235187 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:07.235291 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:07.236877 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.237593 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.239246 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.239640 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.239751 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:07.243618 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:07.243623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:07.243626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:07.243662 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:07.246394 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:07.247220 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:07.247225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:07.247228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:07.247344 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:07.247753 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:07.247756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:07.247758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:07.248093 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:07.248232 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:07.248967 (+ 735us) server_negotiation.cc:299] Negotiation successful
0929 02:52:07.249203 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:52:07.250638 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:07.232295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:38399 (local address 127.8.57.1:49153)
0929 02:52:07.232987 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:07.233025 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:07.234842 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:07.235548 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:07.235557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:07.236103 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:07.236698 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:07.236717 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.237745 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.237748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:07.239105 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:07.239119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.239856 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.239863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:07.240162 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:07.240846 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:07.240876 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:07.243387 (+ 2511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:07.246547 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:07.246557 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:07.246576 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:07.247095 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:07.247448 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:07.247454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:07.247458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:07.247643 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:07.248219 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:07.248227 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:07.248580 (+ 353us) client_negotiation.cc:769] Sending connection context
0929 02:52:07.248821 (+ 241us) client_negotiation.cc:240] Negotiation successful
0929 02:52:07.249386 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":269,"threads_started":1}
I20260929 02:52:07.252249 9183 heartbeater.cc:347] Connected to a master server at 127.8.57.62:38399
I20260929 02:52:07.252535 9183 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:07.253257 9183 heartbeater.cc:511] Master 127.8.57.62:38399 requested a full tablet report, sending...
I20260929 02:52:07.255285 8994 ts_manager.cc:194] Registered new tserver with Master: c18ec7d8741b4794b9c393bd44a36dd5 (127.8.57.1:40843)
I20260929 02:52:07.256788 8994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:49153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:07.271565 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:38399
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:07.576539 9189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:07.576831 9189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:07.576974 9189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:07.584309 9189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:07.584450 9189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:07.584604 9189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:07.594305 9189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:38399
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:07.595583 9189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:07.596962 9189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:07.609205 9197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:07.609229 9194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:07.609256 9195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:07.610651 9189 server_base.cc:1034] running on GCE node
I20260929 02:52:07.611220 9189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:07.612267 9189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:07.613466 9189 hybrid_clock.cc:648] HybridClock initialized: now 1790650327613415 us; error 48 us; skew 500 ppm
I20260929 02:52:07.613757 9189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:07 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650327, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:07.616693 9189 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:07.618083 9189 webserver.cc:460] Webserver started at http://127.8.57.2:37939/ using document root <none> and password file <none>
I20260929 02:52:07.618738 9189 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:07.618847 9189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:07.619130 9189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:07.621963 9189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "3710fcb7cea24423834f8d9a01cf0d83"
format_stamp: "Formatted at 2026-09-29 02:52:07 on dist-test-slave-1dkt"
server_key: "ef53cb4bb94be3b3cd81868b3724330f"
server_key_iv: "ffe5937499d0a618da40e36d68bf2090"
server_key_version: "encryptionkey@0"
I20260929 02:52:07.622706 9189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "3710fcb7cea24423834f8d9a01cf0d83"
format_stamp: "Formatted at 2026-09-29 02:52:07 on dist-test-slave-1dkt"
server_key: "ef53cb4bb94be3b3cd81868b3724330f"
server_key_iv: "ffe5937499d0a618da40e36d68bf2090"
server_key_version: "encryptionkey@0"
I20260929 02:52:07.627666 9189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260929 02:52:07.631040 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:07.632315 9189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:07.632464 9189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "3710fcb7cea24423834f8d9a01cf0d83"
format_stamp: "Formatted at 2026-09-29 02:52:07 on dist-test-slave-1dkt"
server_key: "ef53cb4bb94be3b3cd81868b3724330f"
server_key_iv: "ffe5937499d0a618da40e36d68bf2090"
server_key_version: "encryptionkey@0"
I20260929 02:52:07.632647 9189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:07.645249 9189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:07.649502 9189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:07.649775 9189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:07.650702 9189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:07.652086 9189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:07.652158 9189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:07.652266 9189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:07.652318 9189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:07.681806 9189 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:45125
I20260929 02:52:07.681847 9316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:45125 every 8 connection(s)
I20260929 02:52:07.683038 9189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:07.686357 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9189
I20260929 02:52:07.686555 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:07.687100 8420 external_mini_cluster.cc:1428] Setting key c579e1619361c999e7abaca11d0e1925
Sep 29 02:52:07 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650327, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:07.705423 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:07.685416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:35063 (local address 127.8.57.62:38399)
0929 02:52:07.685584 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:07.685588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:07.686840 (+ 1252us) server_negotiation.cc:407] Connection header received
0929 02:52:07.688976 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:07.688982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:07.689055 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:07.689145 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:07.691066 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.692201 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.694136 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.694560 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.694666 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:07.698819 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:07.698826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:07.698830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:07.698868 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:07.702291 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:07.703179 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:07.703185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:07.703189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:07.703367 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:07.703831 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:07.703835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:07.703838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:07.704190 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:07.704307 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:07.705016 (+ 709us) server_negotiation.cc:299] Negotiation successful
0929 02:52:07.705194 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
I20260929 02:52:07.706486 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:07.685889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:38399 (local address 127.8.57.2:35063)
0929 02:52:07.686661 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:07.686709 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:07.688723 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:07.689403 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:07.689415 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:07.690120 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:07.690870 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:07.690894 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.692352 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.692357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:07.693990 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:07.694006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:07.694766 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:07.694773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:07.695062 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:07.695788 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:07.695816 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:07.698574 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:07.702436 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:07.702447 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:07.702465 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:07.703046 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:07.703474 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:07.703479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:07.703483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:07.703715 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:07.704303 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:07.704310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:07.704636 (+ 326us) client_negotiation.cc:769] Sending connection context
0929 02:52:07.704906 (+ 270us) client_negotiation.cc:240] Negotiation successful
0929 02:52:07.705308 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":277,"threads_started":1}
I20260929 02:52:07.707927 9317 heartbeater.cc:347] Connected to a master server at 127.8.57.62:38399
I20260929 02:52:07.708216 9317 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:07.708900 9317 heartbeater.cc:511] Master 127.8.57.62:38399 requested a full tablet report, sending...
I20260929 02:52:07.710266 8994 ts_manager.cc:194] Registered new tserver with Master: 3710fcb7cea24423834f8d9a01cf0d83 (127.8.57.2:45125)
I20260929 02:52:07.711320 8994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:35063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:07.730247 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:38399
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:08.038290 9322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:08.038568 9322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:08.038659 9322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:08.045953 9322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:08.046063 9322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:08.046183 9322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:08.056011 9322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:38399
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:08.057232 9322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:08.058467 9322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:08.070894 9330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:08.071003 9327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:08.070992 9328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:08.077315 9322 server_base.cc:1034] running on GCE node
I20260929 02:52:08.077935 9322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:08.078899 9322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:08.080089 9322 hybrid_clock.cc:648] HybridClock initialized: now 1790650328080032 us; error 54 us; skew 500 ppm
I20260929 02:52:08.080446 9322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:08.083318 9322 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:08.084952 9322 webserver.cc:460] Webserver started at http://127.8.57.3:45889/ using document root <none> and password file <none>
I20260929 02:52:08.085630 9322 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:08.085748 9322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:08.086082 9322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:08.088951 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "1514ee56cd1f485493ca8bd763f58667"
format_stamp: "Formatted at 2026-09-29 02:52:08 on dist-test-slave-1dkt"
server_key: "1b2a1d441eadafa3f35137d6615fdfd7"
server_key_iv: "25e6735a3bbf2ff4a38d899a92d025c2"
server_key_version: "encryptionkey@0"
I20260929 02:52:08.089694 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "1514ee56cd1f485493ca8bd763f58667"
format_stamp: "Formatted at 2026-09-29 02:52:08 on dist-test-slave-1dkt"
server_key: "1b2a1d441eadafa3f35137d6615fdfd7"
server_key_iv: "25e6735a3bbf2ff4a38d899a92d025c2"
server_key_version: "encryptionkey@0"
I20260929 02:52:08.094671 9322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:52:08.098001 9337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:08.099221 9322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:08.099372 9322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "1514ee56cd1f485493ca8bd763f58667"
format_stamp: "Formatted at 2026-09-29 02:52:08 on dist-test-slave-1dkt"
server_key: "1b2a1d441eadafa3f35137d6615fdfd7"
server_key_iv: "25e6735a3bbf2ff4a38d899a92d025c2"
server_key_version: "encryptionkey@0"
I20260929 02:52:08.099558 9322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:08.121632 9322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:08.125478 9322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:08.125741 9322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:08.126678 9322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:08.128010 9322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:08.128088 9322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:08.128206 9322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:08.128257 9322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:08.159258 9322 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:35745
I20260929 02:52:08.159299 9449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:35745 every 8 connection(s)
I20260929 02:52:08.160526 9322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:08.165776 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9322
I20260929 02:52:08.165982 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:08.166512 8420 external_mini_cluster.cc:1428] Setting key 3100376e34878589d97b1dfc4b75f5fd
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:08.179605 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.162817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:57995 (local address 127.8.57.62:38399)
0929 02:52:08.162995 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.162999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.164129 (+ 1130us) server_negotiation.cc:407] Connection header received
0929 02:52:08.166059 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.166065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.166145 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.166229 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.168090 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.169121 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.170718 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.171145 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.171237 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.174472 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.174477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.174480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.174516 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.176977 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.177619 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.177622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.177625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.177730 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.178061 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.178064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.178066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.178339 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.178452 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.179113 (+ 661us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.179271 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:52:08.180718 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.163252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:38399 (local address 127.8.57.3:57995)
0929 02:52:08.163977 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.164016 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:08.165731 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:08.166485 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:08.166497 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:08.167145 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:08.167918 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.167953 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.169282 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.169287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.170593 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.170605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.171322 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.171326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.171527 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.172175 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:08.172197 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:08.174253 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:08.177099 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:08.177105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:08.177118 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:08.177528 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:08.177801 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:08.177804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:08.177806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:08.177985 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:08.178442 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:08.178448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:08.178761 (+ 313us) client_negotiation.cc:769] Sending connection context
0929 02:52:08.179001 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 02:52:08.179442 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":277,"threads_started":1}
I20260929 02:52:08.182142 9450 heartbeater.cc:347] Connected to a master server at 127.8.57.62:38399
I20260929 02:52:08.182396 9450 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:08.183034 9450 heartbeater.cc:511] Master 127.8.57.62:38399 requested a full tablet report, sending...
I20260929 02:52:08.184377 8993 ts_manager.cc:194] Registered new tserver with Master: 1514ee56cd1f485493ca8bd763f58667 (127.8.57.3:35745)
I20260929 02:52:08.185240 8993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:57995
I20260929 02:52:08.193666 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:52:08.210937 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.199548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.8.57.62:38399)
0929 02:52:08.199759 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.199766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.200094 (+ 328us) server_negotiation.cc:407] Connection header received
0929 02:52:08.200227 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.200231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.200293 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.200371 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.201707 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.202408 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.203617 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.204171 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.204262 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.206433 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.206437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.206440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.206473 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.208820 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.209380 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.209383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.209386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.209493 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.209748 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.209751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.209753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.210051 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.210234 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.210481 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.210650 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:52:08.216953 8993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:08.223559 8993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:08.245019 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.235672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.8.57.1:40843)
0929 02:52:08.236330 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.236338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.236463 (+ 125us) server_negotiation.cc:407] Connection header received
0929 02:52:08.236795 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.236803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.237056 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.237235 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:08.238896 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.240290 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.242111 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.243524 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.243642 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.243856 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.244524 (+ 668us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.244690 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":343,"threads_started":1}
I20260929 02:52:08.245057 9462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.235411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40843 (local address 127.0.0.1:42720)
0929 02:52:08.236316 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.236350 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:08.236529 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:08.237422 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:08.238225 (+ 803us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:08.238266 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:08.238707 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.238720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.240459 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.240464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.241967 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.241978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.244027 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.244034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.244265 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.244313 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 02:52:08.244469 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 02:52:08.244661 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":24,"thread_start_us":324,"threads_started":1}
I20260929 02:52:08.246863 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.237678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.8.57.3:35745)
0929 02:52:08.238335 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.238343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.238860 (+ 517us) server_negotiation.cc:407] Connection header received
0929 02:52:08.239136 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.239144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.239368 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.239515 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:08.240674 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.242199 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.244334 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.245785 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.245901 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.246116 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.246392 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.246555 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":363,"threads_started":1}
I20260929 02:52:08.246893 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.237294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:35745 (local address 127.0.0.1:53878)
0929 02:52:08.238772 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.238786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:08.238965 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:08.239841 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:08.239846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:08.239920 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:08.240431 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.240440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.242373 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.242376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.244197 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.244211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.245983 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.245990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.246148 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.246179 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:52:08.246306 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 02:52:08.246435 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1363,"mutex_wait_us":26,"spinlock_wait_cycles":3840,"thread_start_us":1213,"threads_started":1}
I20260929 02:52:08.247332 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.236544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:45125 (local address 127.0.0.1:55418)
0929 02:52:08.237156 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.237172 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:08.237268 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:08.239844 (+ 2576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:08.239848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:08.239902 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:08.240422 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.240435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.242315 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.242320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.244438 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.244451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.246322 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.246328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.246503 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.246533 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:52:08.246655 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 02:52:08.246785 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":20,"spinlock_wait_cycles":1792,"thread_start_us":462,"threads_started":1}
I20260929 02:52:08.247938 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.236872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.8.57.2:45125)
0929 02:52:08.239202 (+ 2330us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.239209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.239233 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:52:08.239310 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.239316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.239507 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.239665 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:08.240652 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.242199 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.244593 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.245967 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.246080 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.246249 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.246980 (+ 731us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.247148 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2184,"thread_start_us":2093,"threads_started":1}
I20260929 02:52:08.249557 9118 tablet_service.cc:1467] Processing CreateTablet for tablet 595dcdba25b44123a6657e0e136e8507 (DEFAULT_TABLE table=test-table [id=713f1437db174afcaf6636d2e54cf624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:08.250440 9385 tablet_service.cc:1467] Processing CreateTablet for tablet 595dcdba25b44123a6657e0e136e8507 (DEFAULT_TABLE table=test-table [id=713f1437db174afcaf6636d2e54cf624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:08.251066 9118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 595dcdba25b44123a6657e0e136e8507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:08.250978 9252 tablet_service.cc:1467] Processing CreateTablet for tablet 595dcdba25b44123a6657e0e136e8507 (DEFAULT_TABLE table=test-table [id=713f1437db174afcaf6636d2e54cf624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:08.251787 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 595dcdba25b44123a6657e0e136e8507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:08.252048 9252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 595dcdba25b44123a6657e0e136e8507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:08.259791 9183 heartbeater.cc:503] Master 127.8.57.62:38399 was elected leader, sending a full tablet report...
I20260929 02:52:08.264595 9468 tablet_bootstrap.cc:492] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: Bootstrap starting.
I20260929 02:52:08.268581 9468 tablet_bootstrap.cc:654] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:08.269593 9468 log.cc:824] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:08.270526 9470 tablet_bootstrap.cc:492] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: Bootstrap starting.
I20260929 02:52:08.271152 9469 tablet_bootstrap.cc:492] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: Bootstrap starting.
I20260929 02:52:08.274154 9470 tablet_bootstrap.cc:654] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:08.274560 9468 tablet_bootstrap.cc:492] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: No bootstrap required, opened a new log
I20260929 02:52:08.274899 9468 ts_tablet_manager.cc:1397] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260929 02:52:08.274905 9469 tablet_bootstrap.cc:654] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:08.275445 9470 log.cc:824] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:08.275898 9469 log.cc:824] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:08.278574 9470 tablet_bootstrap.cc:492] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: No bootstrap required, opened a new log
I20260929 02:52:08.278827 9469 tablet_bootstrap.cc:492] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: No bootstrap required, opened a new log
I20260929 02:52:08.278849 9470 ts_tablet_manager.cc:1397] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260929 02:52:08.279094 9469 ts_tablet_manager.cc:1397] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20260929 02:52:08.282812 9468 raft_consensus.cc:348] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.283177 9468 raft_consensus.cc:374] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:08.283278 9468 raft_consensus.cc:729] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3710fcb7cea24423834f8d9a01cf0d83, State: Initialized, Role: FOLLOWER
I20260929 02:52:08.283900 9468 consensus_queue.cc:260] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.284950 9317 heartbeater.cc:503] Master 127.8.57.62:38399 was elected leader, sending a full tablet report...
I20260929 02:52:08.285597 9468 ts_tablet_manager.cc:1428] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260929 02:52:08.289036 9470 raft_consensus.cc:348] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.289486 9470 raft_consensus.cc:374] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:08.289582 9470 raft_consensus.cc:729] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1514ee56cd1f485493ca8bd763f58667, State: Initialized, Role: FOLLOWER
I20260929 02:52:08.290191 9470 consensus_queue.cc:260] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.290611 9469 raft_consensus.cc:348] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.291049 9469 raft_consensus.cc:374] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:08.291141 9469 raft_consensus.cc:729] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18ec7d8741b4794b9c393bd44a36dd5, State: Initialized, Role: FOLLOWER
I20260929 02:52:08.291195 9450 heartbeater.cc:503] Master 127.8.57.62:38399 was elected leader, sending a full tablet report...
I20260929 02:52:08.291699 9469 consensus_queue.cc:260] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.291998 9470 ts_tablet_manager.cc:1428] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260929 02:52:08.293366 9469 ts_tablet_manager.cc:1428] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260929 02:52:08.412086 9451 tablet.cc:2369] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:08.415808 9475 raft_consensus.cc:482] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:08.416108 9475 raft_consensus.cc:504] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.417701 9475 leader_election.cc:288] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c18ec7d8741b4794b9c393bd44a36dd5 (127.8.57.1:40843), 3710fcb7cea24423834f8d9a01cf0d83 (127.8.57.2:45125)
I20260929 02:52:08.423771 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.418021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40843 (local address 127.8.57.3:36751)
0929 02:52:08.418213 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.418232 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:08.418368 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:08.418899 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:08.418903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:08.418938 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:08.419105 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.419113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.420349 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.420355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.421776 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.421788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.423051 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.423057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.423238 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.423274 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:52:08.423371 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:52:08.423499 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 02:52:08.423764 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.418136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:36751 (local address 127.8.57.1:40843)
0929 02:52:08.418300 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.418305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.418331 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:52:08.418543 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.418549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.418629 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.418717 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:08.419260 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.420217 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.421935 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.422910 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.423005 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.423078 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.423423 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.423525 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:52:08.424378 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.418654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:40015 (local address 127.8.57.2:45125)
0929 02:52:08.418832 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.418838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.419154 (+ 316us) server_negotiation.cc:407] Connection header received
0929 02:52:08.419278 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.419281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.419337 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.419402 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:08.419914 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.420747 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.422558 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.423472 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.423585 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.423672 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.424024 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.424144 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:52:08.424501 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.418422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:45125 (local address 127.8.57.3:40015)
0929 02:52:08.419064 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.419077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:08.419168 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:08.419432 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:08.419437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:08.419459 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:08.419769 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.419779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.420893 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.420897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.422410 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:08.422423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.423679 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.423685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:08.423851 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.423904 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 02:52:08.423994 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:52:08.424114 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":1,"thread_start_us":339,"threads_started":1}
I20260929 02:52:08.424930 9138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595dcdba25b44123a6657e0e136e8507" candidate_uuid: "1514ee56cd1f485493ca8bd763f58667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" is_pre_election: true
I20260929 02:52:08.425472 9138 raft_consensus.cc:2454] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1514ee56cd1f485493ca8bd763f58667 in term 0.
I20260929 02:52:08.425582 9272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595dcdba25b44123a6657e0e136e8507" candidate_uuid: "1514ee56cd1f485493ca8bd763f58667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3710fcb7cea24423834f8d9a01cf0d83" is_pre_election: true
I20260929 02:52:08.426080 9272 raft_consensus.cc:2454] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1514ee56cd1f485493ca8bd763f58667 in term 0.
I20260929 02:52:08.426155 9340 leader_election.cc:302] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1514ee56cd1f485493ca8bd763f58667, c18ec7d8741b4794b9c393bd44a36dd5; no voters:
I20260929 02:52:08.426499 9475 raft_consensus.cc:2781] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:08.426638 9475 raft_consensus.cc:482] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:08.426738 9475 raft_consensus.cc:3037] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:08.429005 9475 raft_consensus.cc:504] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.429607 9475 leader_election.cc:288] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [CANDIDATE]: Term 1 election: Requested vote from peers c18ec7d8741b4794b9c393bd44a36dd5 (127.8.57.1:40843), 3710fcb7cea24423834f8d9a01cf0d83 (127.8.57.2:45125)
I20260929 02:52:08.430088 9138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595dcdba25b44123a6657e0e136e8507" candidate_uuid: "1514ee56cd1f485493ca8bd763f58667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18ec7d8741b4794b9c393bd44a36dd5"
I20260929 02:52:08.430132 9272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "595dcdba25b44123a6657e0e136e8507" candidate_uuid: "1514ee56cd1f485493ca8bd763f58667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3710fcb7cea24423834f8d9a01cf0d83"
I20260929 02:52:08.430308 9272 raft_consensus.cc:3037] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:08.430310 9138 raft_consensus.cc:3037] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:08.433504 9138 raft_consensus.cc:2454] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1514ee56cd1f485493ca8bd763f58667 in term 1.
I20260929 02:52:08.433497 9272 raft_consensus.cc:2454] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1514ee56cd1f485493ca8bd763f58667 in term 1.
I20260929 02:52:08.433934 9340 leader_election.cc:302] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1514ee56cd1f485493ca8bd763f58667, 3710fcb7cea24423834f8d9a01cf0d83; no voters:
I20260929 02:52:08.434204 9475 raft_consensus.cc:2781] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:08.434691 9475 raft_consensus.cc:686] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [term 1 LEADER]: Becoming Leader. State: Replica: 1514ee56cd1f485493ca8bd763f58667, State: Running, Role: LEADER
W20260929 02:52:08.434849 9318 tablet.cc:2369] T 595dcdba25b44123a6657e0e136e8507 P 3710fcb7cea24423834f8d9a01cf0d83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:08.435173 9475 consensus_queue.cc:237] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } }
I20260929 02:52:08.439733 8994 catalog_manager.cc:5495] T 595dcdba25b44123a6657e0e136e8507 P 1514ee56cd1f485493ca8bd763f58667 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1514ee56cd1f485493ca8bd763f58667 (127.8.57.3). New cstate: current_term: 1 leader_uuid: "1514ee56cd1f485493ca8bd763f58667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c18ec7d8741b4794b9c393bd44a36dd5" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3710fcb7cea24423834f8d9a01cf0d83" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1514ee56cd1f485493ca8bd763f58667" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260929 02:52:08.481513 9184 tablet.cc:2369] T 595dcdba25b44123a6657e0e136e8507 P c18ec7d8741b4794b9c393bd44a36dd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:08.487670 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.473886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.8.57.1:40843)
0929 02:52:08.474104 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.474110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.474433 (+ 323us) server_negotiation.cc:407] Connection header received
0929 02:52:08.474612 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.474616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.474698 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.474783 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.476238 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.477005 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.478239 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.478735 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.478849 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.482885 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.482895 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.482909 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.482955 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.485522 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.486135 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.486142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.486144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.486259 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.486519 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.486521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.486523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.486850 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.487070 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.487262 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.487415 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:08.528533 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.515942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.8.57.1:40843)
0929 02:52:08.516132 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.516138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.516465 (+ 327us) server_negotiation.cc:407] Connection header received
0929 02:52:08.516650 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.516654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.516718 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.516795 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.518165 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.518900 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.520165 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.520569 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.520673 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.524182 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.524187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.524190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.524222 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.526471 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.527012 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.527015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.527017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.527133 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.527400 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.527402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.527404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.527666 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.527840 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.528103 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.528278 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:08.567723 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.554711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.8.57.1:40843)
0929 02:52:08.554990 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.554997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.555286 (+ 289us) server_negotiation.cc:407] Connection header received
0929 02:52:08.555470 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.555475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.555539 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.555623 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.557065 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.557763 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.558956 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.559367 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.559474 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.563014 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.563019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.563022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.563064 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.565499 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.566114 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.566117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.566119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.566229 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.566550 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.566554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.566556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.566888 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.567083 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.567293 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.567474 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
W20260929 02:52:08.569378 9098 tablet_service.cc:2908] Rejecting scan request for tablet 595dcdba25b44123a6657e0e136e8507: Uninitialized: clean time has not yet been initialized
I20260929 02:52:08.572383 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9055
I20260929 02:52:08.583757 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.573159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.8.57.1:40843)
0929 02:52:08.573315 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.573319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.573629 (+ 310us) server_negotiation.cc:407] Connection header received
0929 02:52:08.573744 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.573748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.573814 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.573900 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.574913 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.575591 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.576808 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.577227 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.577327 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.579394 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.579398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.579401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.579436 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.581831 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.582363 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.582366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.582369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.582487 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.582741 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.582744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.582746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.582996 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.583191 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.583365 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.583542 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:52:08.584409 9178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:08.674455 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9055
I20260929 02:52:08.687063 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9189
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:08.704324 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.687869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.8.57.2:45125)
0929 02:52:08.688058 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.688062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.688082 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:52:08.688245 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.688251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.688361 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.688460 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.689868 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.690983 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.692822 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.693440 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.693572 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.698045 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.698061 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.698080 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.698124 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.701651 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.702318 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.702327 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.702331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.702487 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.702805 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.702810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.702812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.703258 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.703563 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.703743 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.703979 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 02:52:08.704973 9312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:08.791119 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9189
I20260929 02:52:08.803634 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9322
Sep 29 02:52:08 dist-test-slave-1dkt krb5kdc[8949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:08.817740 9493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.804342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.8.57.3:35745)
0929 02:52:08.804762 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:08.804766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:08.804782 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:52:08.804836 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:08.804840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:08.804903 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:08.804993 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:08.806130 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.806797 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.808128 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:08.808528 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:08.808619 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:08.812470 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:08.812488 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:08.812509 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:08.812546 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:08.815109 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.815775 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.815783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.815786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.815987 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:08.816324 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:08.816327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:08.816329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:08.816675 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:08.816915 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:08.817181 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 02:52:08.817346 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":224,"threads_started":1}
I20260929 02:52:08.818389 9445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:08.906471 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:08.905974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40843 (local address 127.8.57.3:58893)
0929 02:52:08.906143 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:08.906275 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 02:52:08.906453 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9322
I20260929 02:52:08.918712 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 8962
I20260929 02:52:08.919780 9024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:09.027318 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 8962
2026-09-29T02:52:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3916 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:09 dist-test-slave-1dkt krb5kdc[9498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:09 dist-test-slave-1dkt krb5kdc[9498](info): set up 2 sockets
Sep 29 02:52:09 dist-test-slave-1dkt krb5kdc[9498](info): commencing operation
krb5kdc: starting...
W20260929 02:52:10.015745 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.007s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:10 dist-test-slave-1dkt krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:10.173430 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37945
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:10.470211 9511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:10.470510 9511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:10.470643 9511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:10.477618 9511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:10.477716 9511 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:52:10.477780 9511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:10.477859 9511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:10.477918 9511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:10.477996 9511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:10.487414 9511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37945
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:10.488605 9511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:10.489842 9511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:10.500500 9519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:10.500486 9517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:10.500573 9516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:10.501259 9511 server_base.cc:1034] running on GCE node
I20260929 02:52:10.501915 9511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:10.503206 9511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:10.504402 9511 hybrid_clock.cc:648] HybridClock initialized: now 1790650330504345 us; error 37 us; skew 500 ppm
I20260929 02:52:10.504712 9511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:10 dist-test-slave-1dkt krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650330, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:10.507606 9511 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:10.509083 9511 webserver.cc:460] Webserver started at http://127.8.57.62:37963/ using document root <none> and password file <none>
I20260929 02:52:10.509737 9511 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:10.509828 9511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:10.510123 9511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:10.512979 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "4e0fece14f7244c083f1bb9d58d9f963"
format_stamp: "Formatted at 2026-09-29 02:52:10 on dist-test-slave-1dkt"
server_key: "115d93d7b96cea17461c2fe497061c37"
server_key_iv: "b8a201f86242bf009e1edc1fc8acbb1d"
server_key_version: "encryptionkey@0"
I20260929 02:52:10.513759 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "4e0fece14f7244c083f1bb9d58d9f963"
format_stamp: "Formatted at 2026-09-29 02:52:10 on dist-test-slave-1dkt"
server_key: "115d93d7b96cea17461c2fe497061c37"
server_key_iv: "b8a201f86242bf009e1edc1fc8acbb1d"
server_key_version: "encryptionkey@0"
I20260929 02:52:10.518642 9511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:52:10.521989 9526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:10.523231 9511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:10.523378 9511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "4e0fece14f7244c083f1bb9d58d9f963"
format_stamp: "Formatted at 2026-09-29 02:52:10 on dist-test-slave-1dkt"
server_key: "115d93d7b96cea17461c2fe497061c37"
server_key_iv: "b8a201f86242bf009e1edc1fc8acbb1d"
server_key_version: "encryptionkey@0"
I20260929 02:52:10.523572 9511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:10.558928 9511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:10.563234 9511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:10.563475 9511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:10.583598 9511 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:37945
I20260929 02:52:10.583655 9587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:37945 every 8 connection(s)
I20260929 02:52:10.584977 9511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:10.586467 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9511
I20260929 02:52:10.586692 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:10.587481 8420 external_mini_cluster.cc:1428] Setting key 3b77b9fd9346c03d6c3605cebd2c361d
I20260929 02:52:10.589632 9588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:52:10 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:10.602483 9588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: Bootstrap starting.
I20260929 02:52:10.605842 9588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:10.607040 9588 log.cc:824] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:10.609968 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:10.589457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.8.57.62:37945)
0929 02:52:10.590224 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:10.590238 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:10.590285 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:52:10.592406 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:10.592432 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:10.592945 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:10.593366 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:10.594613 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:10.596626 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:10.597900 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:10.598870 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:10.599057 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:10.603239 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:10.603246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:10.603261 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:10.603299 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:10.606112 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:10.606874 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:10.606880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:10.606886 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:10.607038 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:10.607411 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:10.607414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:10.607416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:10.608144 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:10.608402 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:10.608749 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 02:52:10.609044 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":339,"threads_started":1}
I20260929 02:52:10.610121 9588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: No bootstrap required, opened a new log
I20260929 02:52:10.617165 9588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0fece14f7244c083f1bb9d58d9f963" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37945 } }
I20260929 02:52:10.617482 9588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:10.617576 9588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0fece14f7244c083f1bb9d58d9f963, State: Initialized, Role: FOLLOWER
I20260929 02:52:10.618166 9588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0fece14f7244c083f1bb9d58d9f963" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37945 } }
I20260929 02:52:10.618405 9588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:10.618541 9588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:10.618717 9588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:10.620782 9588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0fece14f7244c083f1bb9d58d9f963" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37945 } }
I20260929 02:52:10.621237 9588 leader_election.cc:302] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e0fece14f7244c083f1bb9d58d9f963; no voters:
I20260929 02:52:10.621745 9588 leader_election.cc:288] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:10.621845 9593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:10.622468 9593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0fece14f7244c083f1bb9d58d9f963, State: Running, Role: LEADER
I20260929 02:52:10.622963 9593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0fece14f7244c083f1bb9d58d9f963" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37945 } }
I20260929 02:52:10.623575 9588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:10.625530 9595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e0fece14f7244c083f1bb9d58d9f963. Latest consensus state: current_term: 1 leader_uuid: "4e0fece14f7244c083f1bb9d58d9f963" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0fece14f7244c083f1bb9d58d9f963" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37945 } } }
I20260929 02:52:10.625499 9594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e0fece14f7244c083f1bb9d58d9f963" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0fece14f7244c083f1bb9d58d9f963" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37945 } } }
I20260929 02:52:10.625919 9594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:10.625919 9595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:10.627571 9599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:10.634967 9599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:10.642971 9599 catalog_manager.cc:1347] Generated new cluster ID: d29252f8b222427d877db868ba58275b
I20260929 02:52:10.643087 9599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:10.662925 9599 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:10.664860 9599 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:10.670889 9599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: Generated new TSK 0
I20260929 02:52:10.671995 9599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:10.721794 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37945
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:11.025406 9614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:11.025689 9614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:11.025836 9614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:11.033172 9614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:11.033298 9614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:11.033425 9614 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:52:11.033535 9614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:11.043977 9614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:11.045310 9614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:11.046618 9614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:11.060350 9619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:11.060263 9620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:11.060307 9622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:11.061123 9614 server_base.cc:1034] running on GCE node
I20260929 02:52:11.061686 9614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:11.062538 9614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:11.063793 9614 hybrid_clock.cc:648] HybridClock initialized: now 1790650331063708 us; error 50 us; skew 500 ppm
I20260929 02:52:11.064119 9614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:11 dist-test-slave-1dkt krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650331, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:11.067202 9614 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:11.068593 9614 webserver.cc:460] Webserver started at http://127.8.57.1:36255/ using document root <none> and password file <none>
I20260929 02:52:11.069226 9614 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:11.069308 9614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:11.069569 9614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:11.072607 9614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "799227d0895042a9af46ff3f1274736a"
format_stamp: "Formatted at 2026-09-29 02:52:11 on dist-test-slave-1dkt"
server_key: "cbca45f892b23ea44feeda6fb72972b4"
server_key_iv: "df4e7e3327b3747e04d53aa6904b0437"
server_key_version: "encryptionkey@0"
I20260929 02:52:11.073320 9614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "799227d0895042a9af46ff3f1274736a"
format_stamp: "Formatted at 2026-09-29 02:52:11 on dist-test-slave-1dkt"
server_key: "cbca45f892b23ea44feeda6fb72972b4"
server_key_iv: "df4e7e3327b3747e04d53aa6904b0437"
server_key_version: "encryptionkey@0"
I20260929 02:52:11.078104 9614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 02:52:11.081281 9629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:11.082499 9614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:52:11.082625 9614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "799227d0895042a9af46ff3f1274736a"
format_stamp: "Formatted at 2026-09-29 02:52:11 on dist-test-slave-1dkt"
server_key: "cbca45f892b23ea44feeda6fb72972b4"
server_key_iv: "df4e7e3327b3747e04d53aa6904b0437"
server_key_version: "encryptionkey@0"
I20260929 02:52:11.082800 9614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:11.138901 9614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:11.142792 9614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:11.143023 9614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:11.145521 9614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:11.145599 9614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:11.145721 9614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:11.145772 9614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:52:11 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650331, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:11.167215 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.146240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:50501 (local address 127.8.57.62:37945)
0929 02:52:11.146713 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:11.146719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:11.147944 (+ 1225us) server_negotiation.cc:407] Connection header received
0929 02:52:11.150398 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:11.150403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:11.150488 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:11.150590 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:11.152688 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.153732 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.156071 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.156727 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.156871 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.160733 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:11.160739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:11.160743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:11.160787 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:11.163905 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.164872 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.164876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.164879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.164997 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.165463 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.165466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.165468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.165734 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:11.165887 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:11.166766 (+ 879us) server_negotiation.cc:299] Negotiation successful
0929 02:52:11.166956 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":237,"threads_started":1}
I20260929 02:52:11.169112 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.146900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.8.57.1:50501)
0929 02:52:11.147742 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:11.147789 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:11.150067 (+ 2278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:11.150878 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:11.150892 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:11.151630 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:11.152522 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.152550 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.153895 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.153901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.155908 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.155928 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.156974 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.156981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.157159 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.157783 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:11.157804 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:11.160457 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:11.164043 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.164054 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.164074 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.164702 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.165093 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.165099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.165103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.165351 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.165848 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:11.165857 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:11.166349 (+ 492us) client_negotiation.cc:769] Sending connection context
0929 02:52:11.166668 (+ 319us) client_negotiation.cc:240] Negotiation successful
0929 02:52:11.167262 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":383,"threads_started":1}
I20260929 02:52:11.193113 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.179010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:58475 (local address 127.8.57.62:37945)
0929 02:52:11.179189 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:11.179196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:11.179795 (+ 599us) server_negotiation.cc:407] Connection header received
0929 02:52:11.180036 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:11.180042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:11.180126 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:11.180227 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:11.181518 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.182210 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.183549 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.184040 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.184168 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.186673 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:11.186678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:11.186682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:11.186712 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:11.190022 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.190644 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.190650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.190655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.190817 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.191131 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.191137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.191140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.191500 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:11.191694 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:11.192348 (+ 654us) server_negotiation.cc:299] Negotiation successful
0929 02:52:11.192694 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:52:11.193125 9711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.178969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.8.57.1:58475)
0929 02:52:11.179677 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:11.179698 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:11.179860 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:11.180274 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:11.180279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:11.180919 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:11.181369 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.181381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.182346 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.182352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.183416 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.183429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.184266 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.184274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.184489 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.185238 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:11.185264 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:11.186485 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:11.190154 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.190160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.190164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.190540 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.190919 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.190922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.190925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.191046 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.191663 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:11.191694 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:11.192005 (+ 311us) client_negotiation.cc:769] Sending connection context
0929 02:52:11.192156 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 02:52:11.192572 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":364,"threads_started":1}
I20260929 02:52:11.200531 9614 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:40607
I20260929 02:52:11.200562 9749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:40607 every 8 connection(s)
I20260929 02:52:11.201710 9614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:11.203642 9750 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37945
I20260929 02:52:11.203908 9750 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:11.204537 9750 heartbeater.cc:511] Master 127.8.57.62:37945 requested a full tablet report, sending...
I20260929 02:52:11.206437 9543 ts_manager.cc:194] Registered new tserver with Master: 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607)
I20260929 02:52:11.207912 9543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:50501
I20260929 02:52:11.208614 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9614
I20260929 02:52:11.208824 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:11.209579 8420 external_mini_cluster.cc:1428] Setting key e1e06fd2b898148e65c4f0459d03589e
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:11.244872 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37945
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:11.550495 9754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:11.550768 9754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:11.550868 9754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:11.557950 9754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:11.558058 9754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:11.558141 9754 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:52:11.558220 9754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:11.567771 9754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:11.569012 9754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:11.570281 9754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:11.582412 9759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:11.582583 9760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:11.582623 9762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:11.583328 9754 server_base.cc:1034] running on GCE node
I20260929 02:52:11.584578 9754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:11.585677 9754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:11.586910 9754 hybrid_clock.cc:648] HybridClock initialized: now 1790650331586846 us; error 38 us; skew 500 ppm
I20260929 02:52:11.587235 9754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:11 dist-test-slave-1dkt krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650331, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:11.590891 9754 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:11.592483 9754 webserver.cc:460] Webserver started at http://127.8.57.2:34341/ using document root <none> and password file <none>
I20260929 02:52:11.593277 9754 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:11.593377 9754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:11.593693 9754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:11.597486 9754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "8b063472c91846829a8d8af214a1d76f"
format_stamp: "Formatted at 2026-09-29 02:52:11 on dist-test-slave-1dkt"
server_key: "16adff899e6cdb277a334cdf2b58e9d2"
server_key_iv: "35daead3572c7cc6b4f1314fcb24b0ed"
server_key_version: "encryptionkey@0"
I20260929 02:52:11.598417 9754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "8b063472c91846829a8d8af214a1d76f"
format_stamp: "Formatted at 2026-09-29 02:52:11 on dist-test-slave-1dkt"
server_key: "16adff899e6cdb277a334cdf2b58e9d2"
server_key_iv: "35daead3572c7cc6b4f1314fcb24b0ed"
server_key_version: "encryptionkey@0"
I20260929 02:52:11.604440 9754 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260929 02:52:11.607825 9769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:11.609166 9754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:11.609309 9754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "8b063472c91846829a8d8af214a1d76f"
format_stamp: "Formatted at 2026-09-29 02:52:11 on dist-test-slave-1dkt"
server_key: "16adff899e6cdb277a334cdf2b58e9d2"
server_key_iv: "35daead3572c7cc6b4f1314fcb24b0ed"
server_key_version: "encryptionkey@0"
I20260929 02:52:11.609548 9754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:11.651612 9754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:11.655710 9754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:11.655987 9754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:11.658581 9754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:11.658684 9754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:11.658810 9754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:11.658872 9754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:52:11 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650331, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:11.681708 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.659374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:50799 (local address 127.8.57.62:37945)
0929 02:52:11.659587 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:11.659594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:11.661176 (+ 1582us) server_negotiation.cc:407] Connection header received
0929 02:52:11.663474 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:11.663481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:11.663573 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:11.663651 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:11.666172 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.667311 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.669512 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.670126 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.670241 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.674872 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:11.674877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:11.674881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:11.674913 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:11.678045 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.678993 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.678998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.679002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.679152 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.679743 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.679748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.679752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.680135 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:11.680294 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:11.681273 (+ 979us) server_negotiation.cc:299] Negotiation successful
0929 02:52:11.681446 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:52:11.683279 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.660167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.8.57.2:50799)
0929 02:52:11.661004 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:11.661049 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:11.663107 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:11.664268 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:11.664276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:11.665093 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:11.665959 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.665995 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.667493 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.667499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.669304 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.669319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.670379 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.670390 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.670650 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.671508 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:11.671544 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:11.674641 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:11.678199 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.678208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.678222 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.678858 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.679274 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.679278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.679285 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.679590 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.680286 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:11.680296 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:11.680829 (+ 533us) client_negotiation.cc:769] Sending connection context
0929 02:52:11.681150 (+ 321us) client_negotiation.cc:240] Negotiation successful
0929 02:52:11.681748 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":371,"threads_started":1}
I20260929 02:52:11.703356 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.689397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.8.57.2:37397)
0929 02:52:11.689986 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:11.690002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:11.690145 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:11.690445 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:11.690449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:11.691012 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:11.691439 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.691453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.692455 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.692461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.694101 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:11.694116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.695095 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.695100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:11.695218 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.695951 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:11.695976 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:11.697042 (+ 1066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:11.700796 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.700800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.700803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.701206 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.701603 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:11.701607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:11.701609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:11.701739 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:11.702402 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:11.702405 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:11.702572 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 02:52:11.702699 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 02:52:11.702951 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":308,"threads_started":1}
I20260929 02:52:11.703374 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:11.689306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:37397 (local address 127.8.57.62:37945)
0929 02:52:11.689456 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:11.689460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:11.690083 (+ 623us) server_negotiation.cc:407] Connection header received
0929 02:52:11.690261 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:11.690266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:11.690324 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:11.690400 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:11.691591 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.692324 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.694222 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:11.694736 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:11.694853 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:11.697240 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:11.697247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:11.697253 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:11.697295 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:11.700647 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.701303 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.701308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.701312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.701465 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:11.701829 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:11.701834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:11.701837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:11.702284 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:11.702447 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:11.702797 (+ 350us) server_negotiation.cc:299] Negotiation successful
0929 02:52:11.703031 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:52:11.706045 9754 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:36719
I20260929 02:52:11.706053 9888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:36719 every 8 connection(s)
I20260929 02:52:11.707325 9754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:11.709069 9889 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37945
I20260929 02:52:11.709241 9889 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:11.709789 9889 heartbeater.cc:511] Master 127.8.57.62:37945 requested a full tablet report, sending...
I20260929 02:52:11.710420 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9754
I20260929 02:52:11.710608 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:11.711027 8420 external_mini_cluster.cc:1428] Setting key 3c87d5a3b446f10d501966f50172c3f8
I20260929 02:52:11.711061 9543 ts_manager.cc:194] Registered new tserver with Master: 8b063472c91846829a8d8af214a1d76f (127.8.57.2:36719)
I20260929 02:52:11.711905 9543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:50799
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:11.745090 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37945
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:12.049810 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:12.050076 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:12.050212 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:12.057348 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:12.057447 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:12.057575 9893 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:52:12.057674 9893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:12.067147 9893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:37945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:12.068413 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:12.069684 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:12.084386 9898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:12.084560 9899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:12.084540 9901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:12.085171 9893 server_base.cc:1034] running on GCE node
I20260929 02:52:12.085706 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:12.086652 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:12.087854 9893 hybrid_clock.cc:648] HybridClock initialized: now 1790650332087777 us; error 54 us; skew 500 ppm
I20260929 02:52:12.088152 9893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:12 dist-test-slave-1dkt krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:12.091048 9893 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:12.092437 9893 webserver.cc:460] Webserver started at http://127.8.57.3:41939/ using document root <none> and password file <none>
I20260929 02:52:12.093068 9893 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:12.093154 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:12.093442 9893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:12.096259 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25"
format_stamp: "Formatted at 2026-09-29 02:52:12 on dist-test-slave-1dkt"
server_key: "264600c9dd53a3fcd5f5bf4048d50cdd"
server_key_iv: "6f9895134f12949b7d544b70270084d7"
server_key_version: "encryptionkey@0"
I20260929 02:52:12.096992 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25"
format_stamp: "Formatted at 2026-09-29 02:52:12 on dist-test-slave-1dkt"
server_key: "264600c9dd53a3fcd5f5bf4048d50cdd"
server_key_iv: "6f9895134f12949b7d544b70270084d7"
server_key_version: "encryptionkey@0"
I20260929 02:52:12.101740 9893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:52:12.104861 9908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:12.106073 9893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:52:12.106212 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25"
format_stamp: "Formatted at 2026-09-29 02:52:12 on dist-test-slave-1dkt"
server_key: "264600c9dd53a3fcd5f5bf4048d50cdd"
server_key_iv: "6f9895134f12949b7d544b70270084d7"
server_key_version: "encryptionkey@0"
I20260929 02:52:12.106393 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:12.136091 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:12.139776 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:12.140061 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:12.142588 9893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:12.142684 9893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:12.142814 9893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:12.142946 9893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 29 02:52:12 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:12.163928 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.143032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:46227 (local address 127.8.57.62:37945)
0929 02:52:12.143503 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.143510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.144651 (+ 1141us) server_negotiation.cc:407] Connection header received
0929 02:52:12.146565 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.146570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.146654 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.146742 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:12.148707 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.149658 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.152068 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.152693 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.152826 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.157519 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:12.157525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:12.157530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:12.157575 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:12.160569 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.161432 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.161436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.161438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.161578 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.162030 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.162033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.162035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.162288 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:12.162392 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.163299 (+ 907us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.163539 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260929 02:52:12.165460 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.143676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.8.57.3:46227)
0929 02:52:12.144452 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.144521 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.146292 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.146974 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.146984 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.147662 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:12.148533 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.148562 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.149880 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.149885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.151909 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.151928 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.152964 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.152975 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.153219 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.154001 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:12.154040 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:12.157252 (+ 3212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:12.160699 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:12.160709 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:12.160728 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:12.161324 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:12.161665 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:12.161670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:12.161674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:12.161930 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:12.162394 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:12.162401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:12.162860 (+ 459us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.163164 (+ 304us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.163763 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":373,"threads_started":1}
I20260929 02:52:12.184545 9893 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:35125
I20260929 02:52:12.184564 10027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:35125 every 8 connection(s)
I20260929 02:52:12.185901 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.171506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.8.57.3:40845)
0929 02:52:12.172206 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.172225 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.172376 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.172734 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.172739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.173320 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:12.173698 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.173711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.175047 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.175050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.176288 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.176302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.177110 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.177122 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.177307 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.178481 (+ 1174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:12.178508 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:12.179499 (+ 991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:12.182931 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:12.182936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:12.182939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:12.183438 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:12.183921 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:12.183926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:12.183934 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:12.184102 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:12.184979 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:12.184982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:12.185132 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.185239 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.185471 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":395,"threads_started":1}
I20260929 02:52:12.186049 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.171273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:40845 (local address 127.8.57.62:37945)
0929 02:52:12.171422 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.171426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.172317 (+ 891us) server_negotiation.cc:407] Connection header received
0929 02:52:12.172524 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.172530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.172605 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.172699 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:12.173842 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.174909 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.176419 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.176828 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.178122 (+ 1294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.179730 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:12.179734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:12.179738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:12.179768 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:12.182810 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.183570 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.183576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.183579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.183724 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.184517 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.184533 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.184536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.184880 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:12.185018 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.185404 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.185633 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 02:52:12.186187 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:12.188242 10028 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37945
I20260929 02:52:12.188457 10028 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:12.189073 10028 heartbeater.cc:511] Master 127.8.57.62:37945 requested a full tablet report, sending...
I20260929 02:52:12.190392 9542 ts_manager.cc:194] Registered new tserver with Master: 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125)
I20260929 02:52:12.190654 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 9893
I20260929 02:52:12.190804 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:12.191241 8420 external_mini_cluster.cc:1428] Setting key 0c6c2ae3f77989d6ffdf956a62ff26f7
I20260929 02:52:12.191258 9542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:46227
I20260929 02:52:12.193851 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:52:12.211180 9750 heartbeater.cc:503] Master 127.8.57.62:37945 was elected leader, sending a full tablet report...
Sep 29 02:52:12 dist-test-slave-1dkt krb5kdc[9498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:52:12 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:12.234349 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.221778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.8.57.62:37945)
0929 02:52:12.221932 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.221936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.222130 (+ 194us) server_negotiation.cc:407] Connection header received
0929 02:52:12.222287 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.222291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.222352 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.222425 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:12.223730 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.224550 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.225812 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.226219 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.226310 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.229781 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:12.229786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:12.229789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:12.229820 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:12.232390 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.232938 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.232942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.232944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.233064 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.233332 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.233334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.233336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.233568 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:12.233745 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.233941 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.234092 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:52:12.238318 9542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:12.245149 9542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:12.276510 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.256784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:35125 (local address 127.0.0.1:40264)
0929 02:52:12.267127 (+ 10343us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.267180 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.267387 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.270110 (+ 2723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.270115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.270144 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:12.270692 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.270702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.272242 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.272247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.274211 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.274223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.275521 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.275528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.275719 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.275775 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.275934 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.276116 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10069,"mutex_wait_us":69,"thread_start_us":11902,"threads_started":1}
I20260929 02:52:12.277401 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.257048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.8.57.3:35125)
0929 02:52:12.269605 (+ 12557us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.269613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.269642 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:52:12.269735 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.269742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.269914 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.270086 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:12.270872 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.272085 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.274348 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.275344 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.276635 (+ 1291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.276867 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.276936 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.277099 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12409,"thread_start_us":12219,"threads_started":1}
I20260929 02:52:12.277804 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.256916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40607 (local address 127.0.0.1:39138)
0929 02:52:12.268267 (+ 11351us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.268290 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.268459 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.269764 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.269772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.269799 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:12.270374 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.270388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.272315 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.272320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.275065 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.275080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.277032 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.277039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.277191 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.277225 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.277306 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.277439 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11238,"mutex_wait_us":90,"thread_start_us":11097,"threads_started":1}
I20260929 02:52:12.278021 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.257371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.8.57.1:40607)
0929 02:52:12.268614 (+ 11243us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.268623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.268654 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:52:12.268759 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.268767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.269044 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.270268 (+ 1224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:12.270564 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.272153 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.275258 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.276598 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.276709 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.276899 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.277365 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.277640 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11084,"thread_start_us":10953,"threads_started":1}
I20260929 02:52:12.278566 10041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.257239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:36719 (local address 127.0.0.1:47824)
0929 02:52:12.268911 (+ 11672us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.268931 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.269114 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.269940 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.269944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.270009 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:12.270563 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.270580 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.272346 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.272354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.275070 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.275080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.277780 (+ 2700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.277787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.277974 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.278005 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.278086 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.278201 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11559,"mutex_wait_us":47,"spinlock_wait_cycles":896,"thread_start_us":9475,"threads_started":1}
I20260929 02:52:12.279244 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.257732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47824 (local address 127.8.57.2:36719)
0929 02:52:12.267829 (+ 10097us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.267852 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.269021 (+ 1169us) server_negotiation.cc:407] Connection header received
0929 02:52:12.269292 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.269299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.269506 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.269679 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:12.270738 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.272216 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.275221 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.277306 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.277451 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.277630 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.278829 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.278965 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9926,"thread_start_us":9803,"threads_started":1}
I20260929 02:52:12.281100 9958 tablet_service.cc:1467] Processing CreateTablet for tablet b2b301b8f2584a459cfcc4b4ca83210f (DEFAULT_TABLE table=test-table [id=64a23b7db5a74319a4fd941e3c6913ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:12.281208 9680 tablet_service.cc:1467] Processing CreateTablet for tablet b2b301b8f2584a459cfcc4b4ca83210f (DEFAULT_TABLE table=test-table [id=64a23b7db5a74319a4fd941e3c6913ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:12.282404 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b301b8f2584a459cfcc4b4ca83210f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.282413 9958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b301b8f2584a459cfcc4b4ca83210f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.284051 9819 tablet_service.cc:1467] Processing CreateTablet for tablet b2b301b8f2584a459cfcc4b4ca83210f (DEFAULT_TABLE table=test-table [id=64a23b7db5a74319a4fd941e3c6913ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:12.285353 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b301b8f2584a459cfcc4b4ca83210f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.294235 10047 tablet_bootstrap.cc:492] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Bootstrap starting.
I20260929 02:52:12.297191 10047 tablet_bootstrap.cc:654] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.298046 10048 tablet_bootstrap.cc:492] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Bootstrap starting.
I20260929 02:52:12.298832 10047 log.cc:824] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:12.299700 10049 tablet_bootstrap.cc:492] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Bootstrap starting.
I20260929 02:52:12.302047 10048 tablet_bootstrap.cc:654] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.303217 10047 tablet_bootstrap.cc:492] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: No bootstrap required, opened a new log
I20260929 02:52:12.303282 10048 log.cc:824] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:12.303308 10049 tablet_bootstrap.cc:654] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.303517 10047 ts_tablet_manager.cc:1397] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:52:12.304410 10049 log.cc:824] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:12.306259 10048 tablet_bootstrap.cc:492] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: No bootstrap required, opened a new log
I20260929 02:52:12.306527 10048 ts_tablet_manager.cc:1397] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260929 02:52:12.306716 10049 tablet_bootstrap.cc:492] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: No bootstrap required, opened a new log
I20260929 02:52:12.306941 10049 ts_tablet_manager.cc:1397] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:52:12.313850 10047 raft_consensus.cc:348] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.314322 10047 raft_consensus.cc:374] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.314464 10047 raft_consensus.cc:729] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799227d0895042a9af46ff3f1274736a, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.314487 10049 raft_consensus.cc:348] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.314913 10049 raft_consensus.cc:374] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.315001 10049 raft_consensus.cc:729] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b063472c91846829a8d8af214a1d76f, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.315104 10047 consensus_queue.cc:260] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.315514 10049 consensus_queue.cc:260] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.316812 10047 ts_tablet_manager.cc:1428] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260929 02:52:12.317644 9889 heartbeater.cc:503] Master 127.8.57.62:37945 was elected leader, sending a full tablet report...
I20260929 02:52:12.318174 10049 ts_tablet_manager.cc:1428] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260929 02:52:12.318807 10048 raft_consensus.cc:348] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.319200 10048 raft_consensus.cc:374] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.319298 10048 raft_consensus.cc:729] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40bcd1e7ae3d4359a676f3fc3e8f0b25, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.319900 10048 consensus_queue.cc:260] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.321434 10048 ts_tablet_manager.cc:1428] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260929 02:52:12.321539 10028 heartbeater.cc:503] Master 127.8.57.62:37945 was elected leader, sending a full tablet report...
I20260929 02:52:12.380990 10054 raft_consensus.cc:482] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:12.381265 10054 raft_consensus.cc:504] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.382799 10054 leader_election.cc:288] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125), 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607)
I20260929 02:52:12.389694 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.383186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:40765 (local address 127.8.57.1:40607)
0929 02:52:12.383405 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.383412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.383941 (+ 529us) server_negotiation.cc:407] Connection header received
0929 02:52:12.384225 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.384230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.384328 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.384449 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:12.385101 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.386144 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.388105 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.388845 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.388960 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.389062 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.389343 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.389474 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:52:12.389766 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.383186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:35215 (local address 127.8.57.3:35125)
0929 02:52:12.383374 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.383380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.383957 (+ 577us) server_negotiation.cc:407] Connection header received
0929 02:52:12.384161 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.384166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.384248 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.384342 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:12.385008 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.386114 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.388156 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.388902 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.389010 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.389111 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.389392 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.389523 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:52:12.389876 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.383058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40607 (local address 127.8.57.2:40765)
0929 02:52:12.383772 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.383792 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.384048 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.384462 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.384470 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.384541 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:12.384983 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.384994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.386304 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.386311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.387970 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.387982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.389075 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.389083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.389205 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.389231 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.389308 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.389433 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":4,"spinlock_wait_cycles":87168,"thread_start_us":346,"threads_started":1}
I20260929 02:52:12.390192 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.383058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:35125 (local address 127.8.57.2:35215)
0929 02:52:12.383807 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.383822 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.383982 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.384377 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.384382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.384409 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:12.384834 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.384851 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.386249 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.386253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.388023 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.388036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.389112 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.389118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.389261 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.389282 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.389355 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.389476 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":88,"thread_start_us":450,"threads_started":1}
I20260929 02:52:12.390815 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b301b8f2584a459cfcc4b4ca83210f" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799227d0895042a9af46ff3f1274736a" is_pre_election: true
I20260929 02:52:12.391053 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b301b8f2584a459cfcc4b4ca83210f" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" is_pre_election: true
I20260929 02:52:12.391278 9700 raft_consensus.cc:2454] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 0.
I20260929 02:52:12.391496 9978 raft_consensus.cc:2454] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 0.
I20260929 02:52:12.391908 9772 leader_election.cc:302] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 799227d0895042a9af46ff3f1274736a, 8b063472c91846829a8d8af214a1d76f; no voters:
I20260929 02:52:12.392318 10054 raft_consensus.cc:2781] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:12.392453 10054 raft_consensus.cc:482] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:12.392551 10054 raft_consensus.cc:3037] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.394639 10054 raft_consensus.cc:504] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.395138 10054 leader_election.cc:288] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 election: Requested vote from peers 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125), 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607)
I20260929 02:52:12.395561 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b301b8f2584a459cfcc4b4ca83210f" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25"
I20260929 02:52:12.395591 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b301b8f2584a459cfcc4b4ca83210f" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799227d0895042a9af46ff3f1274736a"
I20260929 02:52:12.395718 9978 raft_consensus.cc:3037] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.395766 9700 raft_consensus.cc:3037] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.397804 9978 raft_consensus.cc:2454] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 1.
I20260929 02:52:12.397902 9700 raft_consensus.cc:2454] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 1.
I20260929 02:52:12.398157 9773 leader_election.cc:302] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40bcd1e7ae3d4359a676f3fc3e8f0b25, 8b063472c91846829a8d8af214a1d76f; no voters:
I20260929 02:52:12.398437 10054 raft_consensus.cc:2781] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:12.398854 10054 raft_consensus.cc:686] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 1 LEADER]: Becoming Leader. State: Replica: 8b063472c91846829a8d8af214a1d76f, State: Running, Role: LEADER
I20260929 02:52:12.399315 10054 consensus_queue.cc:237] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } }
I20260929 02:52:12.403409 9542 catalog_manager.cc:5495] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b063472c91846829a8d8af214a1d76f (127.8.57.2). New cstate: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } health_report { overall_health: HEALTHY } } }
W20260929 02:52:12.437762 10029 tablet.cc:2369] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:12.439234 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.434066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.8.57.2:36719)
0929 02:52:12.434285 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.434291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.434368 (+ 77us) server_negotiation.cc:407] Connection header received
0929 02:52:12.434621 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.434625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.434701 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.434811 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:12.435105 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.435782 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.437050 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.437452 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.437582 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.438167 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:12.438755 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:12.438857 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.438966 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.439036 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:52:12.446781 9700 raft_consensus.cc:1261] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Refusing update from remote peer 8b063472c91846829a8d8af214a1d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:12.446784 9978 raft_consensus.cc:1261] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Refusing update from remote peer 8b063472c91846829a8d8af214a1d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:12.447671 10058 consensus_queue.cc:1035] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:12.448024 10054 consensus_queue.cc:1035] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 02:52:12.453877 9751 tablet.cc:2369] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:12.458021 10061 mvcc.cc:205] Tried to move back new op lower bound from 7334503761693290496 to 7334503761511440384. Current Snapshot: MvccSnapshot[applied={T|T < 7334503761693290496}]
I20260929 02:52:12.461777 10063 mvcc.cc:205] Tried to move back new op lower bound from 7334503761693290496 to 7334503761511440384. Current Snapshot: MvccSnapshot[applied={T|T < 7334503761693290496}]
I20260929 02:52:12.474804 9542 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:12.475086 9542 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:12.478552 9542 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: Sending DeleteTablet for 3 replicas of tablet b2b301b8f2584a459cfcc4b4ca83210f
I20260929 02:52:12.479619 9680 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b301b8f2584a459cfcc4b4ca83210f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:12 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:39138
I20260929 02:52:12.479635 9958 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b301b8f2584a459cfcc4b4ca83210f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:12 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:40264
I20260929 02:52:12.480351 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b301b8f2584a459cfcc4b4ca83210f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:12 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:47824
I20260929 02:52:12.480525 10070 tablet_replica.cc:330] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: stopping tablet replica
I20260929 02:52:12.480930 10070 raft_consensus.cc:2229] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:12.481464 10070 raft_consensus.cc:2258] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:12.482134 9542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:12.483006 9542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:12.491035 10072 tablet_replica.cc:330] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: stopping tablet replica
I20260929 02:52:12.491463 10072 raft_consensus.cc:2229] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:12.492146 10072 raft_consensus.cc:2258] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:12.492666 9818 tablet_service.cc:1467] Processing CreateTablet for tablet 225707edd72c408e8e3e71bd0f660739 (DEFAULT_TABLE table=test-table [id=89df1ce83b1c4a35b7178198a895050f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:12.493145 9818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225707edd72c408e8e3e71bd0f660739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.493156 10071 tablet_replica.cc:330] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: stopping tablet replica
I20260929 02:52:12.493533 10071 raft_consensus.cc:2229] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:12.493995 10071 raft_consensus.cc:2258] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:12.494315 10072 ts_tablet_manager.cc:1905] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:12.495956 10071 ts_tablet_manager.cc:1905] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:12.498667 9958 tablet_service.cc:1467] Processing CreateTablet for tablet 225707edd72c408e8e3e71bd0f660739 (DEFAULT_TABLE table=test-table [id=89df1ce83b1c4a35b7178198a895050f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:12.498793 9680 tablet_service.cc:1467] Processing CreateTablet for tablet 225707edd72c408e8e3e71bd0f660739 (DEFAULT_TABLE table=test-table [id=89df1ce83b1c4a35b7178198a895050f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:12.499221 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225707edd72c408e8e3e71bd0f660739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.499437 10070 ts_tablet_manager.cc:1905] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:12.501179 10049 tablet_bootstrap.cc:492] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Bootstrap starting.
I20260929 02:52:12.502717 10049 tablet_bootstrap.cc:654] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.505174 10071 ts_tablet_manager.cc:1918] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:12.505327 10071 log.cc:1196] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/b2b301b8f2584a459cfcc4b4ca83210f
I20260929 02:52:12.505791 10071 ts_tablet_manager.cc:1939] T b2b301b8f2584a459cfcc4b4ca83210f P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Deleting consensus metadata
I20260929 02:52:12.505926 10070 ts_tablet_manager.cc:1918] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:12.506067 10070 log.cc:1196] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/b2b301b8f2584a459cfcc4b4ca83210f
I20260929 02:52:12.506258 10047 tablet_bootstrap.cc:492] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Bootstrap starting.
I20260929 02:52:12.506611 10070 ts_tablet_manager.cc:1939] T b2b301b8f2584a459cfcc4b4ca83210f P 799227d0895042a9af46ff3f1274736a: Deleting consensus metadata
I20260929 02:52:12.507757 10047 tablet_bootstrap.cc:654] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.508389 9958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 225707edd72c408e8e3e71bd0f660739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.508520 9530 catalog_manager.cc:4841] TS 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125): tablet b2b301b8f2584a459cfcc4b4ca83210f (table test-table [id=64a23b7db5a74319a4fd941e3c6913ab]) successfully deleted
I20260929 02:52:12.508919 9529 catalog_manager.cc:4841] TS 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607): tablet b2b301b8f2584a459cfcc4b4ca83210f (table test-table [id=64a23b7db5a74319a4fd941e3c6913ab]) successfully deleted
I20260929 02:52:12.511209 10049 tablet_bootstrap.cc:492] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: No bootstrap required, opened a new log
I20260929 02:52:12.511387 10049 ts_tablet_manager.cc:1397] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260929 02:52:12.512327 10049 raft_consensus.cc:348] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.512630 10049 raft_consensus.cc:374] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.512771 10049 raft_consensus.cc:729] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b063472c91846829a8d8af214a1d76f, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.513105 10049 consensus_queue.cc:260] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.513851 10049 ts_tablet_manager.cc:1428] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:52:12.514533 10048 tablet_bootstrap.cc:492] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Bootstrap starting.
I20260929 02:52:12.514914 10072 ts_tablet_manager.cc:1918] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:12.515056 10072 log.cc:1196] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/b2b301b8f2584a459cfcc4b4ca83210f
I20260929 02:52:12.515579 10072 ts_tablet_manager.cc:1939] T b2b301b8f2584a459cfcc4b4ca83210f P 8b063472c91846829a8d8af214a1d76f: Deleting consensus metadata
I20260929 02:52:12.516044 10048 tablet_bootstrap.cc:654] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.517787 9527 catalog_manager.cc:4841] TS 8b063472c91846829a8d8af214a1d76f (127.8.57.2:36719): tablet b2b301b8f2584a459cfcc4b4ca83210f (table test-table [id=64a23b7db5a74319a4fd941e3c6913ab]) successfully deleted
I20260929 02:52:12.520130 10047 tablet_bootstrap.cc:492] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: No bootstrap required, opened a new log
I20260929 02:52:12.520129 10048 tablet_bootstrap.cc:492] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: No bootstrap required, opened a new log
I20260929 02:52:12.520259 10048 ts_tablet_manager.cc:1397] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260929 02:52:12.520318 10047 ts_tablet_manager.cc:1397] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.003s
I20260929 02:52:12.520980 10048 raft_consensus.cc:348] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.521015 10047 raft_consensus.cc:348] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.521209 10047 raft_consensus.cc:374] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.521209 10048 raft_consensus.cc:374] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.521294 10048 raft_consensus.cc:729] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40bcd1e7ae3d4359a676f3fc3e8f0b25, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.521334 10047 raft_consensus.cc:729] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799227d0895042a9af46ff3f1274736a, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.521531 10048 consensus_queue.cc:260] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.521596 10047 consensus_queue.cc:260] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.522053 10048 ts_tablet_manager.cc:1428] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 02:52:12.522131 10047 ts_tablet_manager.cc:1428] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:52:12.619689 10054 raft_consensus.cc:482] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:12.619903 10054 raft_consensus.cc:504] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.620554 10054 leader_election.cc:288] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607), 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125)
I20260929 02:52:12.621167 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225707edd72c408e8e3e71bd0f660739" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799227d0895042a9af46ff3f1274736a" is_pre_election: true
I20260929 02:52:12.621156 9977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225707edd72c408e8e3e71bd0f660739" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" is_pre_election: true
I20260929 02:52:12.621418 9977 raft_consensus.cc:2454] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 0.
I20260929 02:52:12.621426 9700 raft_consensus.cc:2454] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 0.
I20260929 02:52:12.621881 9772 leader_election.cc:302] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 799227d0895042a9af46ff3f1274736a, 8b063472c91846829a8d8af214a1d76f; no voters:
I20260929 02:52:12.622179 10054 raft_consensus.cc:2781] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:12.622318 10054 raft_consensus.cc:482] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:12.622428 10054 raft_consensus.cc:3037] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.625674 10054 raft_consensus.cc:504] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.626397 10054 leader_election.cc:288] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 election: Requested vote from peers 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607), 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125)
I20260929 02:52:12.626873 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225707edd72c408e8e3e71bd0f660739" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799227d0895042a9af46ff3f1274736a"
I20260929 02:52:12.627112 9700 raft_consensus.cc:3037] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.627091 9977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "225707edd72c408e8e3e71bd0f660739" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25"
I20260929 02:52:12.627301 9977 raft_consensus.cc:3037] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.629544 9700 raft_consensus.cc:2454] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 1.
I20260929 02:52:12.629984 9772 leader_election.cc:302] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 799227d0895042a9af46ff3f1274736a, 8b063472c91846829a8d8af214a1d76f; no voters:
I20260929 02:52:12.630254 10054 raft_consensus.cc:2781] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:12.630280 9977 raft_consensus.cc:2454] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 1.
I20260929 02:52:12.630411 10054 raft_consensus.cc:686] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 1 LEADER]: Becoming Leader. State: Replica: 8b063472c91846829a8d8af214a1d76f, State: Running, Role: LEADER
I20260929 02:52:12.630718 10054 consensus_queue.cc:237] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.633370 9541 catalog_manager.cc:5495] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b063472c91846829a8d8af214a1d76f (127.8.57.2). New cstate: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } health_report { overall_health: UNKNOWN } } }
Sep 29 02:52:12 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650330, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:12.699383 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.686267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.8.57.62:37945)
0929 02:52:12.686461 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.686471 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.687073 (+ 602us) server_negotiation.cc:407] Connection header received
0929 02:52:12.687247 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.687252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.687310 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.687383 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:12.688633 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.689414 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.690704 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.691220 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.691326 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.694428 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:12.694433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:12.694436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:12.694464 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:12.696826 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.697461 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.697467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.697471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.697602 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:12.697924 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:12.697929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:12.697932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:12.698193 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:12.698334 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.698715 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.699013 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":75}
I20260929 02:52:12.699409 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.686264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37945 (local address 127.0.0.1:56222)
0929 02:52:12.686974 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.687002 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.687146 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.687428 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.687432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.688152 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:12.688498 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.688508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.689535 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.689540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.690581 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.690594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.691386 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.691393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.691523 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.692175 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:12.692197 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:12.694254 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:12.696939 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:12.696952 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:12.696960 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:12.697357 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:12.697695 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:12.697700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:12.697704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:12.697834 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:12.698314 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:12.698322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:12.698479 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.698613 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.698877 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":398,"threads_started":1}
I20260929 02:52:12.704613 9541 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:56222:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:52:12.705919 9541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 02:52:12.710290 9890 tablet.cc:2369] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:12.713133 9680 tablet_service.cc:1467] Processing CreateTablet for tablet 828a05f648a94ae293fdef125aaf5b09 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f708523b71974731b900765eb8b076dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:52:12.713335 9818 tablet_service.cc:1467] Processing CreateTablet for tablet 828a05f648a94ae293fdef125aaf5b09 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f708523b71974731b900765eb8b076dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:52:12.713694 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828a05f648a94ae293fdef125aaf5b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.713851 9818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828a05f648a94ae293fdef125aaf5b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.718740 10047 tablet_bootstrap.cc:492] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: Bootstrap starting.
I20260929 02:52:12.721778 10049 tablet_bootstrap.cc:492] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: Bootstrap starting.
I20260929 02:52:12.723431 10049 tablet_bootstrap.cc:654] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.730614 9958 tablet_service.cc:1467] Processing CreateTablet for tablet 828a05f648a94ae293fdef125aaf5b09 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f708523b71974731b900765eb8b076dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:52:12.731168 9958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828a05f648a94ae293fdef125aaf5b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:12.732272 10047 tablet_bootstrap.cc:654] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.732849 10049 tablet_bootstrap.cc:492] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: No bootstrap required, opened a new log
I20260929 02:52:12.733026 10049 ts_tablet_manager.cc:1397] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260929 02:52:12.734143 10049 raft_consensus.cc:348] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.735507 10049 raft_consensus.cc:374] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.735657 10049 raft_consensus.cc:729] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b063472c91846829a8d8af214a1d76f, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.736063 10049 consensus_queue.cc:260] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.736763 10049 ts_tablet_manager.cc:1428] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:52:12.736625 10054 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.737824 10054 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.741008 10048 tablet_bootstrap.cc:492] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Bootstrap starting.
I20260929 02:52:12.742714 10048 tablet_bootstrap.cc:654] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:12.747805 10048 tablet_bootstrap.cc:492] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: No bootstrap required, opened a new log
I20260929 02:52:12.748052 10048 ts_tablet_manager.cc:1397] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260929 02:52:12.748647 10047 tablet_bootstrap.cc:492] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: No bootstrap required, opened a new log
I20260929 02:52:12.748668 10048 raft_consensus.cc:348] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.748827 10047 ts_tablet_manager.cc:1397] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.004s
I20260929 02:52:12.748912 10048 raft_consensus.cc:374] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.748994 10048 raft_consensus.cc:729] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40bcd1e7ae3d4359a676f3fc3e8f0b25, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.749228 10048 consensus_queue.cc:260] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.749550 10047 raft_consensus.cc:348] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.749641 10055 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.749866 10047 raft_consensus.cc:374] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:12.749943 10047 raft_consensus.cc:729] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799227d0895042a9af46ff3f1274736a, State: Initialized, Role: FOLLOWER
I20260929 02:52:12.749967 10055 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.750779 10047 consensus_queue.cc:260] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.751299 10053 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.751677 10053 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.752369 10048 ts_tablet_manager.cc:1428] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260929 02:52:12.752375 10047 ts_tablet_manager.cc:1428] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:12.847802 10054 raft_consensus.cc:482] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:12.848076 10054 raft_consensus.cc:504] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.848729 10054 leader_election.cc:288] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607), 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125)
I20260929 02:52:12.849466 9977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828a05f648a94ae293fdef125aaf5b09" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" is_pre_election: true
I20260929 02:52:12.849468 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828a05f648a94ae293fdef125aaf5b09" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799227d0895042a9af46ff3f1274736a" is_pre_election: true
I20260929 02:52:12.849745 9700 raft_consensus.cc:2454] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 0.
I20260929 02:52:12.849745 9977 raft_consensus.cc:2454] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 0.
I20260929 02:52:12.850221 9773 leader_election.cc:302] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40bcd1e7ae3d4359a676f3fc3e8f0b25, 8b063472c91846829a8d8af214a1d76f; no voters:
I20260929 02:52:12.850550 10054 raft_consensus.cc:2781] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:12.850673 10054 raft_consensus.cc:482] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:12.850785 10054 raft_consensus.cc:3037] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.853979 10054 raft_consensus.cc:504] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.854693 10054 leader_election.cc:288] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 election: Requested vote from peers 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607), 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125)
I20260929 02:52:12.855162 9977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828a05f648a94ae293fdef125aaf5b09" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25"
I20260929 02:52:12.855263 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828a05f648a94ae293fdef125aaf5b09" candidate_uuid: "8b063472c91846829a8d8af214a1d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799227d0895042a9af46ff3f1274736a"
I20260929 02:52:12.855424 9977 raft_consensus.cc:3037] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.855522 9700 raft_consensus.cc:3037] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:12.857779 9700 raft_consensus.cc:2454] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 1.
I20260929 02:52:12.858220 9772 leader_election.cc:302] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 799227d0895042a9af46ff3f1274736a, 8b063472c91846829a8d8af214a1d76f; no voters:
I20260929 02:52:12.858269 9977 raft_consensus.cc:2454] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b063472c91846829a8d8af214a1d76f in term 1.
I20260929 02:52:12.858474 10054 raft_consensus.cc:2781] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:12.858839 10054 raft_consensus.cc:686] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [term 1 LEADER]: Becoming Leader. State: Replica: 8b063472c91846829a8d8af214a1d76f, State: Running, Role: LEADER
I20260929 02:52:12.859154 10054 consensus_queue.cc:237] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } }
I20260929 02:52:12.860855 10058 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: TxnStatusTablet state changed. Reason: New leader 8b063472c91846829a8d8af214a1d76f. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.861193 10058 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:52:12.861900 10087 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:52:12.862147 10087 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:52:12.862423 9542 catalog_manager.cc:5495] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b063472c91846829a8d8af214a1d76f (127.8.57.2). New cstate: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:12.905905 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.901078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:36719 (local address 127.0.0.1:47838)
0929 02:52:12.901289 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.901307 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.901445 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.901799 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.901804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.901816 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:52:12.902006 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.902015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.903036 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.903042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.904073 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.904084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.904765 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.904771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.904931 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.904987 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:52:12.905364 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:52:12.905385 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.905452 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.905601 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:52:12.905906 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.901101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.8.57.2:36719)
0929 02:52:12.901265 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.901271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.901368 (+ 97us) server_negotiation.cc:407] Connection header received
0929 02:52:12.901614 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.901620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.901690 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.901763 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:12.902129 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.902914 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.904198 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.904623 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.904728 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.905147 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:12.905272 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:12.905345 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.905485 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.905562 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:52:12.909528 9977 raft_consensus.cc:1261] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Refusing update from remote peer 8b063472c91846829a8d8af214a1d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:12.909528 9700 raft_consensus.cc:1261] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Refusing update from remote peer 8b063472c91846829a8d8af214a1d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:12.910171 10054 consensus_queue.cc:1035] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:12.910447 10058 consensus_queue.cc:1035] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:12.913170 10053 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: TxnStatusTablet state changed. Reason: New leader 8b063472c91846829a8d8af214a1d76f. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.913450 10053 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.917179 10062 mvcc.cc:205] Tried to move back new op lower bound from 7334503763591913472 to 7334503763393458176. Current Snapshot: MvccSnapshot[applied={T|T < 7334503763591913472}]
I20260929 02:52:12.917340 10058 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.917562 10058 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:52:12.917460 10053 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.917660 10053 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 799227d0895042a9af46ff3f1274736a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.925451 10055 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: TxnStatusTablet state changed. Reason: New leader 8b063472c91846829a8d8af214a1d76f. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.925727 10055 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.927369 10091 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.927824 10091 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 8b063472c91846829a8d8af214a1d76f: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:52:12.929636 10055 tablet_replica.cc:439] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b063472c91846829a8d8af214a1d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 } } peers { permanent_uuid: "8b063472c91846829a8d8af214a1d76f" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36719 } } peers { permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 } } }
I20260929 02:52:12.929860 10055 tablet_replica.cc:442] T 828a05f648a94ae293fdef125aaf5b09 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:12.940979 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.935641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:34685 (local address 127.8.57.2:36719)
0929 02:52:12.935801 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:12.935805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:12.936314 (+ 509us) server_negotiation.cc:407] Connection header received
0929 02:52:12.936476 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:12.936481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:12.936555 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:12.936621 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:12.937042 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.937783 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.939160 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.939775 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.939931 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.940299 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:12.940407 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:12.940479 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:12.940594 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 02:52:12.940666 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 02:52:12.941042 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:12.935612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:36719 (local address 127.8.57.2:34685)
0929 02:52:12.936224 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:12.936238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:12.936370 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:12.936645 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:12.936649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:12.936659 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:52:12.936910 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.936919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.937892 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.937897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.939035 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:12.939048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:12.939952 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:12.939958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:12.940081 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:12.940137 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:52:12.940491 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:52:12.940500 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:52:12.940568 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 02:52:12.940711 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":338,"threads_started":1}
I20260929 02:52:12.950603 9977 raft_consensus.cc:1261] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Refusing update from remote peer 8b063472c91846829a8d8af214a1d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:12.950603 9700 raft_consensus.cc:1261] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Refusing update from remote peer 8b063472c91846829a8d8af214a1d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:12.951259 10058 consensus_queue.cc:1035] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "799227d0895042a9af46ff3f1274736a" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:12.951524 10091 consensus_queue.cc:1035] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "40bcd1e7ae3d4359a676f3fc3e8f0b25" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:13.013396 9541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:13.013626 9541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:13.016588 9541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e0fece14f7244c083f1bb9d58d9f963: Sending DeleteTablet for 3 replicas of tablet 225707edd72c408e8e3e71bd0f660739
I20260929 02:52:13.017314 9680 tablet_service.cc:1514] Processing DeleteTablet for tablet 225707edd72c408e8e3e71bd0f660739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:13 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:39138
I20260929 02:52:13.017565 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet 225707edd72c408e8e3e71bd0f660739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:13 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:47824
I20260929 02:52:13.017786 9958 tablet_service.cc:1514] Processing DeleteTablet for tablet 225707edd72c408e8e3e71bd0f660739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:13 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:40264
I20260929 02:52:13.027725 10102 tablet_replica.cc:330] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: stopping tablet replica
I20260929 02:52:13.028512 10102 raft_consensus.cc:2229] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:13.028651 10100 tablet_replica.cc:330] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: stopping tablet replica
I20260929 02:52:13.028861 10100 raft_consensus.cc:2229] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:13.029043 10102 raft_consensus.cc:2258] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:13.029071 10100 raft_consensus.cc:2258] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:13.030258 10102 ts_tablet_manager.cc:1905] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:13.030750 10100 ts_tablet_manager.cc:1905] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:13.037884 10102 ts_tablet_manager.cc:1918] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:13.038160 10102 log.cc:1196] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/225707edd72c408e8e3e71bd0f660739
I20260929 02:52:13.038641 10102 ts_tablet_manager.cc:1939] T 225707edd72c408e8e3e71bd0f660739 P 40bcd1e7ae3d4359a676f3fc3e8f0b25: Deleting consensus metadata
I20260929 02:52:13.039891 9530 catalog_manager.cc:4841] TS 40bcd1e7ae3d4359a676f3fc3e8f0b25 (127.8.57.3:35125): tablet 225707edd72c408e8e3e71bd0f660739 (table test-table [id=89df1ce83b1c4a35b7178198a895050f]) successfully deleted
I20260929 02:52:13.039856 10100 ts_tablet_manager.cc:1918] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:13.042614 10101 tablet_replica.cc:330] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: stopping tablet replica
I20260929 02:52:13.042630 10100 log.cc:1196] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/225707edd72c408e8e3e71bd0f660739
I20260929 02:52:13.042824 10101 raft_consensus.cc:2229] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:13.043061 10100 ts_tablet_manager.cc:1939] T 225707edd72c408e8e3e71bd0f660739 P 799227d0895042a9af46ff3f1274736a: Deleting consensus metadata
I20260929 02:52:13.043208 10101 raft_consensus.cc:2258] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:13.044206 9529 catalog_manager.cc:4841] TS 799227d0895042a9af46ff3f1274736a (127.8.57.1:40607): tablet 225707edd72c408e8e3e71bd0f660739 (table test-table [id=89df1ce83b1c4a35b7178198a895050f]) successfully deleted
I20260929 02:52:13.044497 10101 ts_tablet_manager.cc:1905] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:13.051592 10101 ts_tablet_manager.cc:1918] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:13.051738 10101 log.cc:1196] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/225707edd72c408e8e3e71bd0f660739
I20260929 02:52:13.052155 10101 ts_tablet_manager.cc:1939] T 225707edd72c408e8e3e71bd0f660739 P 8b063472c91846829a8d8af214a1d76f: Deleting consensus metadata
I20260929 02:52:13.053315 9527 catalog_manager.cc:4841] TS 8b063472c91846829a8d8af214a1d76f (127.8.57.2:36719): tablet 225707edd72c408e8e3e71bd0f660739 (table test-table [id=89df1ce83b1c4a35b7178198a895050f]) successfully deleted
Sep 29 02:52:13 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:13.063316 10104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:13.031156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39142 (local address 127.8.57.1:40607)
0929 02:52:13.046313 (+ 15157us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:13.046319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:13.046340 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:52:13.046397 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:13.046402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:13.046477 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:13.046615 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:13.048440 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.049623 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.051539 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.052214 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.052381 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:13.056492 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:13.056520 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:13.056541 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:13.056591 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:13.060410 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.061111 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.061121 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.061125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.061303 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.061659 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.061664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.061668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.062100 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:13.062390 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:13.062696 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 02:52:13.062929 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15032,"thread_start_us":15592,"threads_started":1}
W20260929 02:52:13.064378 9745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39142
I20260929 02:52:13.080364 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:13.066992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.8.57.62:37945)
0929 02:52:13.067233 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:13.067239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:13.067402 (+ 163us) server_negotiation.cc:407] Connection header received
0929 02:52:13.067576 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:13.067580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:13.067639 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:13.067714 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:13.069304 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.070052 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.071825 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.072444 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.072576 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:13.075318 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:13.075323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:13.075327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:13.075361 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:13.077895 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.078690 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.078696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.078698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.078815 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.079156 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.079159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.079161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.079417 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:13.079611 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:13.079962 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 02:52:13.080126 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260929 02:52:13.080947 9541 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56226
I20260929 02:52:13.083204 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9614
I20260929 02:52:13.112076 10104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:13.084057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.8.57.1:40607)
0929 02:52:13.084274 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:13.084280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:13.099121 (+ 14841us) server_negotiation.cc:407] Connection header received
0929 02:52:13.099360 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:13.099366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:13.099464 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:13.099573 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:13.100903 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.102043 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.103242 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.103941 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.104064 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:13.106180 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:13.106186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:13.106191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:13.106232 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:13.109495 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.110155 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.110161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.110165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.110314 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.110623 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.110628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.110631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.111007 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:13.111248 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:13.111480 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 02:52:13.111718 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 02:52:13.112727 9745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39148
W20260929 02:52:13.113252 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9614: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:13.113453 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9614
I20260929 02:52:13.133316 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9754
Sep 29 02:52:13 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:13.152426 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:13.135700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.8.57.2:36719)
0929 02:52:13.135858 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:13.135862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:13.135896 (+ 34us) server_negotiation.cc:407] Connection header received
0929 02:52:13.135964 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:13.135967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:13.136052 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:13.136170 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:13.137833 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.138954 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.141330 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.141817 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.141916 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:13.146778 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:13.146792 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:13.146806 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:13.146842 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:13.149715 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.150466 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.150473 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.150475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.150594 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.150958 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.150961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.150963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.151310 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:13.151544 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:13.151964 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 02:52:13.152134 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 02:52:13.153350 9884 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47850
W20260929 02:52:13.154017 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9754: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:13.154192 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9754
I20260929 02:52:13.171028 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9893
Sep 29 02:52:13 dist-test-slave-1dkt krb5kdc[9498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650332, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:13.184914 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:13.171692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.8.57.3:35125)
0929 02:52:13.172214 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:13.172218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:13.172233 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:52:13.172286 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:13.172289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:13.172365 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:13.172450 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:13.173533 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.174211 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.175340 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:13.175735 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:13.175827 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:13.179987 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:13.180000 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:13.180017 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:13.180049 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:13.182523 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.183100 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.183107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.183109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.183223 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:13.183494 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:13.183497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:13.183499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:13.183830 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:13.184211 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:13.184476 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 02:52:13.184640 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":308,"threads_started":1}
W20260929 02:52:13.185590 10023 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40270
W20260929 02:52:13.186254 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9893: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:13.186400 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9893
I20260929 02:52:13.198945 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 9511
I20260929 02:52:13.200006 9583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:13.312459 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 9511
2026-09-29T02:52:13Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4285 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:13 dist-test-slave-1dkt krb5kdc[10113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:13 dist-test-slave-1dkt krb5kdc[10113](info): set up 2 sockets
Sep 29 02:52:13 dist-test-slave-1dkt krb5kdc[10113](info): commencing operation
krb5kdc: starting...
W20260929 02:52:14.295954 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:14 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:14.460799 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42099
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:14.761776 10126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:14.762100 10126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:14.762233 10126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:14.769529 10126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:14.769666 10126 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:52:14.769737 10126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:14.769825 10126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:14.769889 10126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:14.769980 10126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:14.780025 10126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42099
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:14.781284 10126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:14.782589 10126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:14.793846 10134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:14.793880 10132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:14.794034 10131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:14.794718 10126 server_base.cc:1034] running on GCE node
I20260929 02:52:14.795507 10126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:14.796962 10126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:14.798203 10126 hybrid_clock.cc:648] HybridClock initialized: now 1790650334798136 us; error 41 us; skew 500 ppm
I20260929 02:52:14.798524 10126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:14 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650334, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:14.801505 10126 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:14.802911 10126 webserver.cc:460] Webserver started at http://127.8.57.62:42813/ using document root <none> and password file <none>
I20260929 02:52:14.803599 10126 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:14.803695 10126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:14.804037 10126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:14.806974 10126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "9523d6c1e2fa49bda91b3b60f289073f"
format_stamp: "Formatted at 2026-09-29 02:52:14 on dist-test-slave-1dkt"
server_key: "1061f559e8ff25c7266a018657c95a2f"
server_key_iv: "9dc1cb7083526400f80086392de0e7e1"
server_key_version: "encryptionkey@0"
I20260929 02:52:14.807684 10126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "9523d6c1e2fa49bda91b3b60f289073f"
format_stamp: "Formatted at 2026-09-29 02:52:14 on dist-test-slave-1dkt"
server_key: "1061f559e8ff25c7266a018657c95a2f"
server_key_iv: "9dc1cb7083526400f80086392de0e7e1"
server_key_version: "encryptionkey@0"
I20260929 02:52:14.812645 10126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:52:14.815841 10141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:14.817143 10126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:14.817293 10126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "9523d6c1e2fa49bda91b3b60f289073f"
format_stamp: "Formatted at 2026-09-29 02:52:14 on dist-test-slave-1dkt"
server_key: "1061f559e8ff25c7266a018657c95a2f"
server_key_iv: "9dc1cb7083526400f80086392de0e7e1"
server_key_version: "encryptionkey@0"
I20260929 02:52:14.817520 10126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:14.834622 10126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:14.838382 10126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:14.838630 10126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:14.858289 10126 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42099
I20260929 02:52:14.858299 10202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42099 every 8 connection(s)
I20260929 02:52:14.859575 10126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:14.863322 10203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:14.865685 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 10126
I20260929 02:52:14.865897 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:14.866660 8420 external_mini_cluster.cc:1428] Setting key 3a4bdf73c2d50fed0c402bac7de37005
I20260929 02:52:14.875566 10203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Bootstrap starting.
I20260929 02:52:14.879263 10203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:14.880504 10203 log.cc:824] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:52:14 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:14.883849 10203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: No bootstrap required, opened a new log
I20260929 02:52:14.889885 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:14.868672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.8.57.62:42099)
0929 02:52:14.869450 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:14.869462 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:14.869507 (+ 45us) server_negotiation.cc:407] Connection header received
0929 02:52:14.871121 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:14.871143 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:14.871504 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:14.871823 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:14.873498 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:14.874746 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:14.876392 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:14.877058 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:14.877202 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:14.881823 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:14.881834 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:14.881853 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:14.881894 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:14.885464 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:14.886163 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:14.886171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:14.886180 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:14.886379 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:14.886703 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:14.886711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:14.886715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:14.887399 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:14.887704 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:14.888061 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 02:52:14.888503 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":360,"threads_started":1}
I20260929 02:52:14.893574 10203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9523d6c1e2fa49bda91b3b60f289073f" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42099 } }
I20260929 02:52:14.893894 10203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:14.893962 10203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9523d6c1e2fa49bda91b3b60f289073f, State: Initialized, Role: FOLLOWER
I20260929 02:52:14.894447 10203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9523d6c1e2fa49bda91b3b60f289073f" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42099 } }
I20260929 02:52:14.894649 10203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:14.894742 10203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:14.894850 10203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:14.896992 10203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9523d6c1e2fa49bda91b3b60f289073f" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42099 } }
I20260929 02:52:14.897451 10203 leader_election.cc:302] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9523d6c1e2fa49bda91b3b60f289073f; no voters:
I20260929 02:52:14.897938 10203 leader_election.cc:288] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:14.898072 10208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:14.899118 10208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [term 1 LEADER]: Becoming Leader. State: Replica: 9523d6c1e2fa49bda91b3b60f289073f, State: Running, Role: LEADER
I20260929 02:52:14.899674 10208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9523d6c1e2fa49bda91b3b60f289073f" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42099 } }
I20260929 02:52:14.900100 10203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:14.902482 10209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9523d6c1e2fa49bda91b3b60f289073f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9523d6c1e2fa49bda91b3b60f289073f" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42099 } } }
I20260929 02:52:14.902890 10209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:14.902737 10210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9523d6c1e2fa49bda91b3b60f289073f. Latest consensus state: current_term: 1 leader_uuid: "9523d6c1e2fa49bda91b3b60f289073f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9523d6c1e2fa49bda91b3b60f289073f" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42099 } } }
I20260929 02:52:14.903009 10210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:14.906028 10217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:14.910611 10217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:14.918493 10217 catalog_manager.cc:1347] Generated new cluster ID: 05e78c4d252748c7bd3850e5a9fb2c98
I20260929 02:52:14.918620 10217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:14.946286 10217 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:14.947629 10217 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:14.956091 10217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Generated new TSK 0
I20260929 02:52:14.956976 10217 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:15.011824 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42099
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:15.343621 10229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:15.343911 10229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:15.344046 10229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:15.351234 10229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:15.351341 10229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:15.351464 10229 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:52:15.351527 10229 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:52:15.351626 10229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:15.361287 10229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:15.362517 10229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:15.363759 10229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:15.375953 10235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:15.375943 10234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:15.376168 10237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:15.376991 10229 server_base.cc:1034] running on GCE node
I20260929 02:52:15.377559 10229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:15.378384 10229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:15.379611 10229 hybrid_clock.cc:648] HybridClock initialized: now 1790650335379560 us; error 52 us; skew 500 ppm
I20260929 02:52:15.379966 10229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:15 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650335, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:15.382951 10229 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:15.384308 10229 webserver.cc:460] Webserver started at http://127.8.57.1:41813/ using document root <none> and password file <none>
I20260929 02:52:15.384938 10229 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:15.385049 10229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:15.385324 10229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:15.388116 10229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "6b214eee827944b1b8385aaffbfbcebc"
format_stamp: "Formatted at 2026-09-29 02:52:15 on dist-test-slave-1dkt"
server_key: "a6d679e34685650751a1f18d2237e3bd"
server_key_iv: "1e482059b0b5cc06d94bc640ef716f40"
server_key_version: "encryptionkey@0"
I20260929 02:52:15.388927 10229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "6b214eee827944b1b8385aaffbfbcebc"
format_stamp: "Formatted at 2026-09-29 02:52:15 on dist-test-slave-1dkt"
server_key: "a6d679e34685650751a1f18d2237e3bd"
server_key_iv: "1e482059b0b5cc06d94bc640ef716f40"
server_key_version: "encryptionkey@0"
I20260929 02:52:15.393805 10229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:15.396996 10244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:15.398257 10229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:15.398401 10229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "6b214eee827944b1b8385aaffbfbcebc"
format_stamp: "Formatted at 2026-09-29 02:52:15 on dist-test-slave-1dkt"
server_key: "a6d679e34685650751a1f18d2237e3bd"
server_key_iv: "1e482059b0b5cc06d94bc640ef716f40"
server_key_version: "encryptionkey@0"
I20260929 02:52:15.398581 10229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:15.413489 10229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:15.417019 10229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:15.417295 10229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:15.419714 10229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:15.419811 10229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:15.419939 10229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:15.420017 10229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:52:15 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650335, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:15.440294 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.420331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:36675 (local address 127.8.57.62:42099)
0929 02:52:15.420793 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:15.420799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:15.421844 (+ 1045us) server_negotiation.cc:407] Connection header received
0929 02:52:15.423848 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:15.423853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:15.423961 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:15.424058 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:15.426046 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.426978 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.429388 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.429872 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.429991 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.434592 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:15.434599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:15.434603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:15.434644 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:15.437417 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.438128 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.438134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.438138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.438254 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.438648 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.438653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.438657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.438922 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:15.439057 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:15.439699 (+ 642us) server_negotiation.cc:299] Negotiation successful
0929 02:52:15.439941 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":230,"threads_started":1}
I20260929 02:52:15.441694 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.420817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.8.57.1:36675)
0929 02:52:15.421680 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:15.421730 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:15.423577 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:15.424301 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:15.424309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:15.424998 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:15.425857 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.425887 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.427130 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.427134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.429249 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.429264 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.430169 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.430178 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.430487 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.431282 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:15.431310 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:15.434305 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:15.437550 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.437559 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.437576 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.438025 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.438350 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.438355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.438359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.438546 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.439037 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:15.439044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:15.439369 (+ 325us) client_negotiation.cc:769] Sending connection context
0929 02:52:15.439602 (+ 233us) client_negotiation.cc:240] Negotiation successful
0929 02:52:15.440133 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":337,"threads_started":1}
I20260929 02:52:15.461735 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.447828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:39879 (local address 127.8.57.62:42099)
0929 02:52:15.448017 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:15.448024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:15.448607 (+ 583us) server_negotiation.cc:407] Connection header received
0929 02:52:15.448769 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:15.448773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:15.448830 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:15.448902 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:15.450040 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.450730 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.452696 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.453229 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.453357 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.455557 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:15.455562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:15.455565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:15.455598 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:15.458811 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.459630 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.459635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.459639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.459787 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.460216 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.460222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.460225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.460596 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:15.460746 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:15.461204 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 02:52:15.461409 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:52:15.461848 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.447791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.8.57.1:39879)
0929 02:52:15.448501 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:15.448521 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:15.448671 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:15.448948 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:15.448953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:15.449550 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:15.449914 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.449926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.450846 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.450851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.452551 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.452566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.453453 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.453459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.453649 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.454250 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:15.454273 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:15.455397 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:15.458929 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.458933 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.458935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.459469 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.459923 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.459927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.459930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.460073 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.460728 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:15.460733 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:15.460941 (+ 208us) client_negotiation.cc:769] Sending connection context
0929 02:52:15.461078 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 02:52:15.461403 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":398,"threads_started":1}
I20260929 02:52:15.468575 10229 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:39803
I20260929 02:52:15.468605 10364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:39803 every 8 connection(s)
I20260929 02:52:15.469782 10229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:15.471578 10365 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42099
I20260929 02:52:15.471774 10365 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:15.472398 10365 heartbeater.cc:511] Master 127.8.57.62:42099 requested a full tablet report, sending...
I20260929 02:52:15.474138 10158 ts_manager.cc:194] Registered new tserver with Master: 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803)
I20260929 02:52:15.475502 10158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:36675
I20260929 02:52:15.478770 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 10229
I20260929 02:52:15.478921 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:15.479465 8420 external_mini_cluster.cc:1428] Setting key 8cfc53c96caf4f2d7b8bdba7081dc997
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:15.514703 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42099
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:15.814477 10369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:15.814728 10369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:15.814862 10369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:15.821924 10369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:15.822019 10369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:15.822140 10369 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:52:15.822197 10369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:52:15.822294 10369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:15.831769 10369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:15.833010 10369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:15.834266 10369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:15.846535 10374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:15.846561 10375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:15.846513 10377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:15.847330 10369 server_base.cc:1034] running on GCE node
I20260929 02:52:15.848009 10369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:15.848968 10369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:15.850219 10369 hybrid_clock.cc:648] HybridClock initialized: now 1790650335850163 us; error 46 us; skew 500 ppm
I20260929 02:52:15.850566 10369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:15 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650335, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:15.853701 10369 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:15.855188 10369 webserver.cc:460] Webserver started at http://127.8.57.2:42261/ using document root <none> and password file <none>
I20260929 02:52:15.855845 10369 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:15.856009 10369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:15.856305 10369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:15.859247 10369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "a3c108aab89944ae90f305129bfce8dc"
format_stamp: "Formatted at 2026-09-29 02:52:15 on dist-test-slave-1dkt"
server_key: "0d8a5086b151ad8518a3a1feead80b0f"
server_key_iv: "e2cab91f9a7fc8fa91f0c4851bad310b"
server_key_version: "encryptionkey@0"
I20260929 02:52:15.860031 10369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "a3c108aab89944ae90f305129bfce8dc"
format_stamp: "Formatted at 2026-09-29 02:52:15 on dist-test-slave-1dkt"
server_key: "0d8a5086b151ad8518a3a1feead80b0f"
server_key_iv: "e2cab91f9a7fc8fa91f0c4851bad310b"
server_key_version: "encryptionkey@0"
I20260929 02:52:15.864970 10369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:52:15.868338 10384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:15.869701 10369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:52:15.869848 10369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "a3c108aab89944ae90f305129bfce8dc"
format_stamp: "Formatted at 2026-09-29 02:52:15 on dist-test-slave-1dkt"
server_key: "0d8a5086b151ad8518a3a1feead80b0f"
server_key_iv: "e2cab91f9a7fc8fa91f0c4851bad310b"
server_key_version: "encryptionkey@0"
I20260929 02:52:15.870031 10369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:15.894276 10369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:15.898061 10369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:15.898315 10369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:15.901031 10369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:15.901137 10369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:15.901260 10369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:15.901352 10369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:52:15 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650335, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:15.921684 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.901387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:44549 (local address 127.8.57.62:42099)
0929 02:52:15.901602 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:15.901608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:15.902975 (+ 1367us) server_negotiation.cc:407] Connection header received
0929 02:52:15.904930 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:15.904937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:15.905028 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:15.905145 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:15.907541 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.908496 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.911205 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.911672 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.911785 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.915294 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:15.915298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:15.915301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:15.915335 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:15.918207 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.919110 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.919113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.919116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.919221 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.919772 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.919776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.919778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.920082 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:15.920228 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:15.921107 (+ 879us) server_negotiation.cc:299] Negotiation successful
0929 02:52:15.921329 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:52:15.922932 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.901926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.8.57.2:44549)
0929 02:52:15.902807 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:15.902857 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:15.904667 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:15.905571 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:15.905585 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:15.906422 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:15.907330 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.907367 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.908689 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.908695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.911027 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.911042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.911915 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.911924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.912138 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.912707 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:15.912728 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:15.915019 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:15.918371 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.918382 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.918404 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.919001 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.919344 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.919351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.919353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.919619 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.920213 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:15.920222 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:15.920683 (+ 461us) client_negotiation.cc:769] Sending connection context
0929 02:52:15.920995 (+ 312us) client_negotiation.cc:240] Negotiation successful
0929 02:52:15.921487 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":355,"threads_started":1}
I20260929 02:52:15.943387 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.927865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60825 (local address 127.8.57.62:42099)
0929 02:52:15.928057 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:15.928063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:15.932327 (+ 4264us) server_negotiation.cc:407] Connection header received
0929 02:52:15.932478 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:15.932483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:15.932547 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:15.932625 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:15.933503 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.934225 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.935459 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.935843 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.935992 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.938209 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:15.938214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:15.938217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:15.938245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:15.940899 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.941481 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.941486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.941490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.941600 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:15.941935 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:15.941939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:15.941943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:15.942354 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:15.942486 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:15.942859 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 02:52:15.943067 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:52:15.943562 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:15.927939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.8.57.2:60825)
0929 02:52:15.932225 (+ 4286us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:15.932244 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:15.932372 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:15.932660 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:15.932664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:15.933106 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:15.933373 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.933382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.934347 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.934351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.935335 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:15.935345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:15.936052 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:15.936059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:15.936251 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:15.936960 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:15.936984 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:15.938037 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:15.941012 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.941018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.941022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.941378 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.941708 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:15.941713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:15.941717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:15.941838 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:15.942449 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:15.942454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:15.942637 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 02:52:15.942762 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 02:52:15.943079 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4164,"thread_start_us":4051,"threads_started":1}
I20260929 02:52:15.953855 10369 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:33223
I20260929 02:52:15.953886 10503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:33223 every 8 connection(s)
I20260929 02:52:15.955001 10369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:15.956909 10504 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42099
I20260929 02:52:15.957109 10504 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:15.957705 10504 heartbeater.cc:511] Master 127.8.57.62:42099 requested a full tablet report, sending...
I20260929 02:52:15.958949 10158 ts_manager.cc:194] Registered new tserver with Master: a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223)
I20260929 02:52:15.959705 10158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:44549
I20260929 02:52:15.962344 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 10369
I20260929 02:52:15.962482 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:15.962900 8420 external_mini_cluster.cc:1428] Setting key 27a07aac9b7b87af32898bd4c0f22125
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:15.998559 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42099
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:16.314800 10508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:16.315065 10508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:16.315198 10508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:16.322392 10508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:16.322511 10508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:16.322633 10508 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:52:16.322697 10508 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:52:16.322798 10508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:16.332753 10508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:42099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:16.333968 10508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:16.335212 10508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:16.347294 10513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:16.347220 10516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:16.347259 10514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:16.348124 10508 server_base.cc:1034] running on GCE node
I20260929 02:52:16.348649 10508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:16.349494 10508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:16.350674 10508 hybrid_clock.cc:648] HybridClock initialized: now 1790650336350627 us; error 44 us; skew 500 ppm
I20260929 02:52:16.350944 10508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:16 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650336, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:16.354040 10508 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:16.355476 10508 webserver.cc:460] Webserver started at http://127.8.57.3:44483/ using document root <none> and password file <none>
I20260929 02:52:16.356192 10508 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:16.356305 10508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:16.356577 10508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:16.359346 10508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "01807f2cf15249bd8d67af3af9be7c2b"
format_stamp: "Formatted at 2026-09-29 02:52:16 on dist-test-slave-1dkt"
server_key: "b58fef3245d5da7e84527292eb850eeb"
server_key_iv: "d1e016a10c467982f39f1cc3c52a30c6"
server_key_version: "encryptionkey@0"
I20260929 02:52:16.360138 10508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "01807f2cf15249bd8d67af3af9be7c2b"
format_stamp: "Formatted at 2026-09-29 02:52:16 on dist-test-slave-1dkt"
server_key: "b58fef3245d5da7e84527292eb850eeb"
server_key_iv: "d1e016a10c467982f39f1cc3c52a30c6"
server_key_version: "encryptionkey@0"
I20260929 02:52:16.365047 10508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:52:16.368340 10523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:16.369570 10508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:52:16.369711 10508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "01807f2cf15249bd8d67af3af9be7c2b"
format_stamp: "Formatted at 2026-09-29 02:52:16 on dist-test-slave-1dkt"
server_key: "b58fef3245d5da7e84527292eb850eeb"
server_key_iv: "d1e016a10c467982f39f1cc3c52a30c6"
server_key_version: "encryptionkey@0"
I20260929 02:52:16.369912 10508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:16.413524 10508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:16.417374 10508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:16.417659 10508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:16.420192 10508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:16.420292 10508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:16.420430 10508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:16.420497 10508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:52:16 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650336, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:16.443487 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.420838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:58387 (local address 127.8.57.62:42099)
0929 02:52:16.421036 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.421043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.422488 (+ 1445us) server_negotiation.cc:407] Connection header received
0929 02:52:16.425365 (+ 2877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.425372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.425466 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.425547 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:16.428314 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.429321 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.431556 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.432238 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.432379 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.436410 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:16.436416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:16.436421 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:16.436464 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:16.439750 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.440875 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.440881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.440885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.441071 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.441666 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.441671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.441675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.442022 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:16.442158 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.442893 (+ 735us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.443139 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 02:52:16.445122 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.421444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.8.57.3:58387)
0929 02:52:16.422315 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.422367 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.424980 (+ 2613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.426164 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.426180 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.427065 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:16.428098 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.428131 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.429471 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.429477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.431362 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.431391 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.432506 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.432515 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.432742 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.433486 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:16.433511 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:16.436142 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:16.439931 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:16.439941 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:16.439963 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:16.440720 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:16.441207 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:16.441211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:16.441213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:16.441494 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:16.442158 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:16.442170 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:16.442524 (+ 354us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.442760 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.443330 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":333,"threads_started":1}
I20260929 02:52:16.464726 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.450861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:33139 (local address 127.8.57.62:42099)
0929 02:52:16.451051 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.451058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.451554 (+ 496us) server_negotiation.cc:407] Connection header received
0929 02:52:16.451762 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.451767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.451831 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.451979 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:16.453234 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.454174 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.455998 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.456401 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.456518 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.458588 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:16.458595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:16.458601 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:16.458643 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:16.462148 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.462762 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.462768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.462772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.462940 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.463238 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.463242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.463246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.463598 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:16.463750 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.464171 (+ 421us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.464377 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:52:16.464843 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.450825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.8.57.3:33139)
0929 02:52:16.451441 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.451468 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.451616 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.452005 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.452010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.452646 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:16.453096 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.453113 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.454333 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.454339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.455813 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.455830 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.456758 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.456764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.456946 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.457655 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:16.457673 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:16.458381 (+ 708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:16.462278 (+ 3897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:16.462282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:16.462285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:16.462659 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:16.463039 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:16.463042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:16.463044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:16.463159 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:16.463689 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:16.463692 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:16.463843 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.464032 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.464382 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":310,"threads_started":1}
I20260929 02:52:16.472478 10508 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:44599
I20260929 02:52:16.472478 10642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:44599 every 8 connection(s)
I20260929 02:52:16.473999 10508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:16.475536 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 10508
I20260929 02:52:16.475687 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:16.476200 8420 external_mini_cluster.cc:1428] Setting key 9fa5c5186ffff054ae7858b8c1af24c1
I20260929 02:52:16.476280 10643 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42099
I20260929 02:52:16.476491 10643 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:16.477175 10643 heartbeater.cc:511] Master 127.8.57.62:42099 requested a full tablet report, sending...
I20260929 02:52:16.477980 10365 heartbeater.cc:503] Master 127.8.57.62:42099 was elected leader, sending a full tablet report...
I20260929 02:52:16.478576 10157 ts_manager.cc:194] Registered new tserver with Master: 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:16.479512 10157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:58387
I20260929 02:52:16.490866 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:52:16 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:52:16 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:16.531838 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.519194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.8.57.62:42099)
0929 02:52:16.519407 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.519414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.519665 (+ 251us) server_negotiation.cc:407] Connection header received
0929 02:52:16.519841 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.519845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.519944 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.520026 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:16.521356 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.522066 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.523252 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.523648 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.523754 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.527233 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:16.527237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:16.527240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:16.527270 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:16.529538 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.530077 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.530082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.530084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.530207 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.530540 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.530544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.530548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.530793 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:16.531032 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.531297 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.531513 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:52:16.535642 10157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:16.542155 10157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:16.563649 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.553152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:33223 (local address 127.0.0.1:56832)
0929 02:52:16.553967 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.553997 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.554143 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.555033 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.555040 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.555070 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:16.555470 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.555483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.557995 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.558001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.560560 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.560576 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.562721 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.562728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.562936 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.562984 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.563110 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.563286 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":448,"threads_started":1}
I20260929 02:52:16.564510 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.553331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.8.57.2:33223)
0929 02:52:16.553981 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.553990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.554083 (+ 93us) server_negotiation.cc:407] Connection header received
0929 02:52:16.554327 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.554335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.554619 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.554802 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:16.555703 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.557807 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.560775 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.562227 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.562359 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.562587 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.564062 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.564240 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":364,"threads_started":1}
I20260929 02:52:16.568720 10434 tablet_service.cc:1467] Processing CreateTablet for tablet c998ba74d0fd4701b2ee95907c916a3e (DEFAULT_TABLE table=test-table [id=adb447ba49274ff1b781dc5af7f6762f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:16.570173 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c998ba74d0fd4701b2ee95907c916a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:16.572049 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.554694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.8.57.3:44599)
0929 02:52:16.564447 (+ 9753us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.564455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.564489 (+ 34us) server_negotiation.cc:407] Connection header received
0929 02:52:16.564586 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.564595 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.564867 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.565084 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:16.566537 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.567912 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.569697 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.571045 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.571154 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.571379 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.571557 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.571716 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9581,"thread_start_us":9405,"threads_started":1}
I20260929 02:52:16.572080 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.554367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:44599 (local address 127.0.0.1:49830)
0929 02:52:16.554869 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.554886 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.555003 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.565054 (+ 10051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.565059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.565083 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:16.566305 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.566317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.568040 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.568045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.569553 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.569567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.571221 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.571227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.571393 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.571426 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.571507 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.571635 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":5,"thread_start_us":250,"threads_started":1}
I20260929 02:52:16.572852 10660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.554944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:39803 (local address 127.0.0.1:36116)
0929 02:52:16.563050 (+ 8106us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.563067 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.563162 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.565253 (+ 2091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.565257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.565279 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:16.565573 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.565583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.567542 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.567547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.569701 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.569715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.571718 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.571724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.571929 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.571960 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.572071 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.572191 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7985,"mutex_wait_us":69,"thread_start_us":8282,"threads_started":1}
I20260929 02:52:16.576333 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.555466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.8.57.1:39803)
0929 02:52:16.564718 (+ 9252us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.564725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.564748 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:52:16.564819 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.564825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.565052 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.565753 (+ 701us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:16.565980 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.567413 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.569897 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.571274 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.571398 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.571578 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.575990 (+ 4412us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.576129 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9109,"thread_start_us":8977,"threads_started":1}
I20260929 02:52:16.575814 10573 tablet_service.cc:1467] Processing CreateTablet for tablet c998ba74d0fd4701b2ee95907c916a3e (DEFAULT_TABLE table=test-table [id=adb447ba49274ff1b781dc5af7f6762f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:16.577220 10573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c998ba74d0fd4701b2ee95907c916a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:16.578991 10295 tablet_service.cc:1467] Processing CreateTablet for tablet c998ba74d0fd4701b2ee95907c916a3e (DEFAULT_TABLE table=test-table [id=adb447ba49274ff1b781dc5af7f6762f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:16.580045 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c998ba74d0fd4701b2ee95907c916a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:16.584463 10662 tablet_bootstrap.cc:492] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Bootstrap starting.
I20260929 02:52:16.588054 10662 tablet_bootstrap.cc:654] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:16.589257 10662 log.cc:824] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:16.589818 10663 tablet_bootstrap.cc:492] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Bootstrap starting.
I20260929 02:52:16.592648 10663 tablet_bootstrap.cc:654] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:16.594120 10663 log.cc:824] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:16.596203 10665 tablet_bootstrap.cc:492] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Bootstrap starting.
I20260929 02:52:16.596901 10663 tablet_bootstrap.cc:492] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: No bootstrap required, opened a new log
I20260929 02:52:16.597127 10663 ts_tablet_manager.cc:1397] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:52:16.598344 10662 tablet_bootstrap.cc:492] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: No bootstrap required, opened a new log
I20260929 02:52:16.598605 10662 ts_tablet_manager.cc:1397] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260929 02:52:16.599454 10665 tablet_bootstrap.cc:654] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:16.600688 10665 log.cc:824] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:16.604250 10665 tablet_bootstrap.cc:492] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: No bootstrap required, opened a new log
I20260929 02:52:16.604522 10665 ts_tablet_manager.cc:1397] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:52:16.604902 10663 raft_consensus.cc:348] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.605283 10663 raft_consensus.cc:374] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:16.605404 10663 raft_consensus.cc:729] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01807f2cf15249bd8d67af3af9be7c2b, State: Initialized, Role: FOLLOWER
I20260929 02:52:16.606015 10663 consensus_queue.cc:260] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.607050 10643 heartbeater.cc:503] Master 127.8.57.62:42099 was elected leader, sending a full tablet report...
I20260929 02:52:16.608817 10663 ts_tablet_manager.cc:1428] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260929 02:52:16.608757 10662 raft_consensus.cc:348] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.609207 10662 raft_consensus.cc:374] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:16.609306 10662 raft_consensus.cc:729] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Initialized, Role: FOLLOWER
I20260929 02:52:16.609902 10662 consensus_queue.cc:260] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.610924 10504 heartbeater.cc:503] Master 127.8.57.62:42099 was elected leader, sending a full tablet report...
I20260929 02:52:16.611572 10662 ts_tablet_manager.cc:1428] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260929 02:52:16.614322 10665 raft_consensus.cc:348] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.614765 10665 raft_consensus.cc:374] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:16.614869 10665 raft_consensus.cc:729] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Initialized, Role: FOLLOWER
I20260929 02:52:16.615489 10665 consensus_queue.cc:260] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.617059 10665 ts_tablet_manager.cc:1428] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260929 02:52:16.706851 10505 tablet.cc:2369] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:52:16.721930 10366 tablet.cc:2369] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:52:16.726449 10644 tablet.cc:2369] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:16.731796 10670 raft_consensus.cc:482] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:16.732100 10670 raft_consensus.cc:504] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.733656 10670 leader_election.cc:288] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:16.739711 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.734024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:54067 (local address 127.8.57.2:33223)
0929 02:52:16.734201 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.734205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.734744 (+ 539us) server_negotiation.cc:407] Connection header received
0929 02:52:16.734928 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.734933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.735012 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.735098 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:16.735637 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.736389 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.738131 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.738887 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.739001 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.739103 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.739351 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.739467 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:52:16.739825 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.734012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:33223 (local address 127.8.57.1:54067)
0929 02:52:16.734634 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.734649 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.734795 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.735179 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.735184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.735210 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:16.735498 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.735507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.736507 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.736512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.737978 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.737989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.739090 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.739095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.739215 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.739239 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.739315 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.739443 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":38,"thread_start_us":326,"threads_started":1}
I20260929 02:52:16.740064 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.734881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:47969 (local address 127.8.57.3:44599)
0929 02:52:16.735083 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.735088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.735397 (+ 309us) server_negotiation.cc:407] Connection header received
0929 02:52:16.735556 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.735560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.735616 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.735685 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:16.736226 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.736914 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.738503 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.739226 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.739333 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.739440 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.739696 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.739811 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:52:16.740305 10672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.734737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:44599 (local address 127.8.57.1:47969)
0929 02:52:16.735298 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.735310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.735428 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.735707 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.735711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.735732 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:16.736086 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.736095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.737041 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.737046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.738359 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.738369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.739423 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.739429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.739563 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.739585 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.739657 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.739778 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":3,"thread_start_us":279,"threads_started":1}
I20260929 02:52:16.740764 10459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c998ba74d0fd4701b2ee95907c916a3e" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c108aab89944ae90f305129bfce8dc" is_pre_election: true
I20260929 02:52:16.741207 10459 raft_consensus.cc:2454] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 0.
I20260929 02:52:16.741122 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c998ba74d0fd4701b2ee95907c916a3e" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b" is_pre_election: true
I20260929 02:52:16.741559 10593 raft_consensus.cc:2454] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 0.
I20260929 02:52:16.741791 10246 leader_election.cc:302] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:16.742157 10670 raft_consensus.cc:2781] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:16.742297 10670 raft_consensus.cc:482] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:16.742398 10670 raft_consensus.cc:3037] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:16.744565 10670 raft_consensus.cc:504] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.745116 10670 leader_election.cc:288] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 election: Requested vote from peers a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:16.745476 10459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c998ba74d0fd4701b2ee95907c916a3e" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c108aab89944ae90f305129bfce8dc"
I20260929 02:52:16.745573 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c998ba74d0fd4701b2ee95907c916a3e" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b"
I20260929 02:52:16.745639 10459 raft_consensus.cc:3037] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:16.745774 10593 raft_consensus.cc:3037] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:16.747797 10459 raft_consensus.cc:2454] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 1.
I20260929 02:52:16.748001 10593 raft_consensus.cc:2454] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 1.
I20260929 02:52:16.748195 10246 leader_election.cc:302] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:16.748446 10670 raft_consensus.cc:2781] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:16.748816 10670 raft_consensus.cc:686] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 1 LEADER]: Becoming Leader. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Running, Role: LEADER
I20260929 02:52:16.749249 10670 consensus_queue.cc:237] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } }
I20260929 02:52:16.753619 10157 catalog_manager.cc:5495] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1). New cstate: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } health_report { overall_health: HEALTHY } } }
Sep 29 02:52:16 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650334, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:16.845402 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.830295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.8.57.62:42099)
0929 02:52:16.830456 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.830461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.830945 (+ 484us) server_negotiation.cc:407] Connection header received
0929 02:52:16.831115 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.831120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.831196 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.831274 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:16.832435 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.833135 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.834315 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.834707 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.834818 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.840669 (+ 5851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:16.840673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:16.840676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:16.840702 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:16.842887 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.843511 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.843533 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.843536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.843670 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:16.844037 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:16.844041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:16.844044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:16.844283 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:16.844403 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.844825 (+ 422us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.845104 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":54}
I20260929 02:52:16.845422 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.830295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42099 (local address 127.0.0.1:34368)
0929 02:52:16.830847 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.830862 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.830995 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.831321 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.831325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.832034 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:16.832304 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.832313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.833249 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.833254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.834193 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.834204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.834867 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.834872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.835023 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.838424 (+ 3401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:16.838441 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:16.840517 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:16.843012 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:16.843023 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:16.843031 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:16.843409 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:16.843765 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:16.843770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:16.843774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:16.843945 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:16.844395 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:16.844403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:16.844574 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.844707 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.844969 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":305,"threads_started":1}
I20260929 02:52:16.850085 10157 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:34368:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:52:16.851048 10157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:16.856973 10295 tablet_service.cc:1467] Processing CreateTablet for tablet 518eed57b0d948f2a82e4e261157c626 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=22b074f153d049e6bee5856c5808a5be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:52:16.857105 10434 tablet_service.cc:1467] Processing CreateTablet for tablet 518eed57b0d948f2a82e4e261157c626 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=22b074f153d049e6bee5856c5808a5be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:52:16.857304 10573 tablet_service.cc:1467] Processing CreateTablet for tablet 518eed57b0d948f2a82e4e261157c626 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=22b074f153d049e6bee5856c5808a5be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:52:16.857936 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518eed57b0d948f2a82e4e261157c626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:16.857983 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518eed57b0d948f2a82e4e261157c626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:16.857983 10573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518eed57b0d948f2a82e4e261157c626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:16.864744 10663 tablet_bootstrap.cc:492] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: Bootstrap starting.
I20260929 02:52:16.864744 10662 tablet_bootstrap.cc:492] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: Bootstrap starting.
I20260929 02:52:16.864746 10665 tablet_bootstrap.cc:492] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: Bootstrap starting.
I20260929 02:52:16.866258 10665 tablet_bootstrap.cc:654] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:16.866286 10662 tablet_bootstrap.cc:654] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:16.866286 10663 tablet_bootstrap.cc:654] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:16.877727 10662 tablet_bootstrap.cc:492] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: No bootstrap required, opened a new log
I20260929 02:52:16.877753 10665 tablet_bootstrap.cc:492] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: No bootstrap required, opened a new log
I20260929 02:52:16.877882 10665 ts_tablet_manager.cc:1397] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260929 02:52:16.877920 10663 tablet_bootstrap.cc:492] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: No bootstrap required, opened a new log
I20260929 02:52:16.877946 10662 ts_tablet_manager.cc:1397] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260929 02:52:16.878046 10663 ts_tablet_manager.cc:1397] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260929 02:52:16.878692 10665 raft_consensus.cc:348] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.878716 10663 raft_consensus.cc:348] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.878927 10663 raft_consensus.cc:374] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:16.878742 10662 raft_consensus.cc:348] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.878965 10665 raft_consensus.cc:374] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:16.879050 10665 raft_consensus.cc:729] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Initialized, Role: FOLLOWER
I20260929 02:52:16.879060 10662 raft_consensus.cc:374] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:16.879072 10663 raft_consensus.cc:729] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01807f2cf15249bd8d67af3af9be7c2b, State: Initialized, Role: FOLLOWER
I20260929 02:52:16.879164 10662 raft_consensus.cc:729] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Initialized, Role: FOLLOWER
I20260929 02:52:16.879302 10663 consensus_queue.cc:260] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.879295 10665 consensus_queue.cc:260] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.879701 10670 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.879738 10668 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.880005 10670 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:16.880070 10668 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:16.880503 10663 ts_tablet_manager.cc:1428] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 02:52:16.880888 10665 ts_tablet_manager.cc:1428] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 02:52:16.879570 10662 consensus_queue.cc:260] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.883559 10662 ts_tablet_manager.cc:1428] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260929 02:52:16.883694 10669 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.884030 10669 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:16.933718 10670 raft_consensus.cc:482] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:16.933925 10670 raft_consensus.cc:504] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.934559 10670 leader_election.cc:288] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:16.935146 10459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518eed57b0d948f2a82e4e261157c626" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c108aab89944ae90f305129bfce8dc" is_pre_election: true
I20260929 02:52:16.935173 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518eed57b0d948f2a82e4e261157c626" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b" is_pre_election: true
I20260929 02:52:16.935395 10459 raft_consensus.cc:2454] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 0.
I20260929 02:52:16.935424 10593 raft_consensus.cc:2454] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 0.
I20260929 02:52:16.935814 10246 leader_election.cc:302] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:16.936127 10670 raft_consensus.cc:2781] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:16.936228 10670 raft_consensus.cc:482] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:16.936323 10670 raft_consensus.cc:3037] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:16.938223 10670 raft_consensus.cc:504] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.938706 10670 leader_election.cc:288] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 election: Requested vote from peers a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:16.939164 10459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518eed57b0d948f2a82e4e261157c626" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c108aab89944ae90f305129bfce8dc"
I20260929 02:52:16.939195 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518eed57b0d948f2a82e4e261157c626" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b"
I20260929 02:52:16.939381 10593 raft_consensus.cc:3037] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:16.939381 10459 raft_consensus.cc:3037] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:16.942171 10593 raft_consensus.cc:2454] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 1.
I20260929 02:52:16.942171 10459 raft_consensus.cc:2454] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 1.
I20260929 02:52:16.942560 10246 leader_election.cc:302] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01807f2cf15249bd8d67af3af9be7c2b, 6b214eee827944b1b8385aaffbfbcebc; no voters:
I20260929 02:52:16.942812 10670 raft_consensus.cc:2781] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:16.942991 10670 raft_consensus.cc:686] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [term 1 LEADER]: Becoming Leader. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Running, Role: LEADER
I20260929 02:52:16.943238 10670 consensus_queue.cc:237] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:16.944844 10673 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: TxnStatusTablet state changed. Reason: New leader 6b214eee827944b1b8385aaffbfbcebc. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.945223 10673 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:52:16.945861 10685 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:52:16.946116 10685 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:52:16.946132 10158 catalog_manager.cc:5495] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1). New cstate: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:16.963758 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.958653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:39803 (local address 127.0.0.1:36126)
0929 02:52:16.958864 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:16.958883 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:16.959034 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:16.959470 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:16.959474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:16.959486 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:52:16.959657 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.959665 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.960670 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.960676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.961640 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:16.961652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.962304 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.962310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:16.962480 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.962529 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:52:16.963247 (+ 718us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:52:16.963267 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 02:52:16.963338 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 02:52:16.963476 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:52:16.963760 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:16.958670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.8.57.1:39803)
0929 02:52:16.958958 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:16.958968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:16.958985 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:16.959218 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:16.959222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:16.959320 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:16.959443 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:16.959776 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.960554 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.961767 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:16.962164 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:16.962266 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:16.962695 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:16.963147 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:16.963270 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:16.963366 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 02:52:16.963453 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260929 02:52:16.971199 10459 raft_consensus.cc:1261] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Refusing update from remote peer 6b214eee827944b1b8385aaffbfbcebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:16.971266 10593 raft_consensus.cc:1261] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Refusing update from remote peer 6b214eee827944b1b8385aaffbfbcebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:16.972170 10673 consensus_queue.cc:1035] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:16.972944 10670 consensus_queue.cc:1035] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:16.983063 10669 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: TxnStatusTablet state changed. Reason: New leader 6b214eee827944b1b8385aaffbfbcebc. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.983397 10669 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:16.985941 10668 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: TxnStatusTablet state changed. Reason: New leader 6b214eee827944b1b8385aaffbfbcebc. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.986191 10668 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:16.986389 10688 mvcc.cc:205] Tried to move back new op lower bound from 7334503780224909312 to 7334503780121935872. Current Snapshot: MvccSnapshot[applied={T|T < 7334503780224909312}]
I20260929 02:52:16.989223 10670 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.989529 10670 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:52:16.989759 10670 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.990001 10670 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 6b214eee827944b1b8385aaffbfbcebc: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:52:16.993395 10690 mvcc.cc:205] Tried to move back new op lower bound from 7334503780224909312 to 7334503780121935872. Current Snapshot: MvccSnapshot[applied={T|T < 7334503780224909312}]
I20260929 02:52:16.996392 10669 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:16.996626 10669 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P a3c108aab89944ae90f305129bfce8dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:17.000362 10668 tablet_replica.cc:439] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } } }
I20260929 02:52:17.000607 10668 tablet_replica.cc:442] T 518eed57b0d948f2a82e4e261157c626 P 01807f2cf15249bd8d67af3af9be7c2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:52:17.005726 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:17.000044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.8.57.1:39803)
0929 02:52:17.000212 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:17.000217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:17.000235 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:52:17.000537 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:17.000542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:17.000618 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:17.000699 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:17.001290 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:17.002452 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:17.004050 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:17.004520 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:17.004650 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:17.005057 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:17.005178 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:17.005269 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:17.005375 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 02:52:17.005452 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:52:17.022492 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:17.016290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:58957 (local address 127.8.57.1:39803)
0929 02:52:17.016446 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:17.016450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:17.016998 (+ 548us) server_negotiation.cc:407] Connection header received
0929 02:52:17.017172 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:17.017178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:17.017253 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:17.017343 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:17.017819 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:17.018947 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:17.020347 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:17.021078 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:17.021214 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:17.021673 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:17.021796 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:17.021888 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:17.022075 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:52:17.022153 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 02:52:17.022588 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:17.016340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:39803 (local address 127.8.57.1:58957)
0929 02:52:17.016893 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:17.016908 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:17.017038 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:17.017351 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:17.017354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:17.017367 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:52:17.017690 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:17.017698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:17.019062 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:17.019065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:17.020228 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:17.020239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:17.021276 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:17.021281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:17.021416 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:17.021470 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:52:17.021918 (+ 448us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:52:17.021929 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 02:52:17.022052 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 02:52:17.022193 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":298,"threads_started":1}
I20260929 02:52:17.032159 10593 raft_consensus.cc:1261] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Refusing update from remote peer 6b214eee827944b1b8385aaffbfbcebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:17.032302 10459 raft_consensus.cc:1261] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Refusing update from remote peer 6b214eee827944b1b8385aaffbfbcebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:17.032711 10700 consensus_queue.cc:1035] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:17.033083 10670 consensus_queue.cc:1035] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:17.038583 10689 mvcc.cc:205] Tried to move back new op lower bound from 7334503780479389696 to 7334503779328503808. Current Snapshot: MvccSnapshot[applied={T|T < 7334503780479389696}]
I20260929 02:52:17.099495 10158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:17.099788 10158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:17.103451 10158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Sending DeleteTablet for 3 replicas of tablet c998ba74d0fd4701b2ee95907c916a3e
I20260929 02:52:17.104471 10434 tablet_service.cc:1514] Processing DeleteTablet for tablet c998ba74d0fd4701b2ee95907c916a3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:17 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:56832
I20260929 02:52:17.104872 10573 tablet_service.cc:1514] Processing DeleteTablet for tablet c998ba74d0fd4701b2ee95907c916a3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:17 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:49830
I20260929 02:52:17.105163 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet c998ba74d0fd4701b2ee95907c916a3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:17 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:36116
I20260929 02:52:17.105789 10705 tablet_replica.cc:330] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: stopping tablet replica
I20260929 02:52:17.105790 10158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:52:17.106503 10705 raft_consensus.cc:2229] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Raft consensus shutting down.
W20260929 02:52:17.106791 10158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:17.107029 10705 raft_consensus.cc:2258] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:17.109123 10705 ts_tablet_manager.cc:1905] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:17.113708 10706 tablet_replica.cc:330] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: stopping tablet replica
I20260929 02:52:17.114162 10706 raft_consensus.cc:2229] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:17.114508 10706 raft_consensus.cc:2258] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:17.115067 10295 tablet_service.cc:1467] Processing CreateTablet for tablet 3e634805ef0b4c6db55b940d73053dca (DEFAULT_TABLE table=test-table [id=0faecfecaa1e408a85ec41f8c9202f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:17.115475 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e634805ef0b4c6db55b940d73053dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:17.117504 10705 ts_tablet_manager.cc:1918] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:17.117720 10705 log.cc:1196] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/c998ba74d0fd4701b2ee95907c916a3e
I20260929 02:52:17.118201 10705 ts_tablet_manager.cc:1939] T c998ba74d0fd4701b2ee95907c916a3e P a3c108aab89944ae90f305129bfce8dc: Deleting consensus metadata
I20260929 02:52:17.120170 10434 tablet_service.cc:1467] Processing CreateTablet for tablet 3e634805ef0b4c6db55b940d73053dca (DEFAULT_TABLE table=test-table [id=0faecfecaa1e408a85ec41f8c9202f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:17.120350 10572 tablet_service.cc:1467] Processing CreateTablet for tablet 3e634805ef0b4c6db55b940d73053dca (DEFAULT_TABLE table=test-table [id=0faecfecaa1e408a85ec41f8c9202f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:17.120504 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e634805ef0b4c6db55b940d73053dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:17.120790 10572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e634805ef0b4c6db55b940d73053dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:17.124282 10143 catalog_manager.cc:4841] TS a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223): tablet c998ba74d0fd4701b2ee95907c916a3e (table test-table [id=adb447ba49274ff1b781dc5af7f6762f]) successfully deleted
I20260929 02:52:17.125156 10662 tablet_bootstrap.cc:492] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Bootstrap starting.
I20260929 02:52:17.126399 10662 tablet_bootstrap.cc:654] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:17.127434 10663 tablet_bootstrap.cc:492] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Bootstrap starting.
I20260929 02:52:17.128955 10663 tablet_bootstrap.cc:654] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:17.129321 10662 tablet_bootstrap.cc:492] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: No bootstrap required, opened a new log
I20260929 02:52:17.129458 10662 ts_tablet_manager.cc:1397] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260929 02:52:17.130128 10662 raft_consensus.cc:348] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.130357 10662 raft_consensus.cc:374] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:17.130432 10662 raft_consensus.cc:729] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Initialized, Role: FOLLOWER
I20260929 02:52:17.130628 10707 tablet_replica.cc:330] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: stopping tablet replica
I20260929 02:52:17.131044 10707 raft_consensus.cc:2229] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:17.131436 10663 tablet_bootstrap.cc:492] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: No bootstrap required, opened a new log
I20260929 02:52:17.131711 10707 raft_consensus.cc:2258] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:17.132771 10663 ts_tablet_manager.cc:1397] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 02:52:17.133584 10663 raft_consensus.cc:348] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.133716 10706 ts_tablet_manager.cc:1905] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:17.133929 10663 raft_consensus.cc:374] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:17.134055 10663 raft_consensus.cc:729] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01807f2cf15249bd8d67af3af9be7c2b, State: Initialized, Role: FOLLOWER
I20260929 02:52:17.134393 10663 consensus_queue.cc:260] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.138221 10663 ts_tablet_manager.cc:1428] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260929 02:52:17.130707 10662 consensus_queue.cc:260] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.139358 10662 ts_tablet_manager.cc:1428] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260929 02:52:17.141000 10706 ts_tablet_manager.cc:1918] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:17.141147 10706 log.cc:1196] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/c998ba74d0fd4701b2ee95907c916a3e
I20260929 02:52:17.141628 10706 ts_tablet_manager.cc:1939] T c998ba74d0fd4701b2ee95907c916a3e P 01807f2cf15249bd8d67af3af9be7c2b: Deleting consensus metadata
I20260929 02:52:17.143301 10143 catalog_manager.cc:4841] TS 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599): tablet c998ba74d0fd4701b2ee95907c916a3e (table test-table [id=adb447ba49274ff1b781dc5af7f6762f]) successfully deleted
I20260929 02:52:17.143455 10665 tablet_bootstrap.cc:492] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Bootstrap starting.
I20260929 02:52:17.144642 10707 ts_tablet_manager.cc:1905] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:17.145061 10665 tablet_bootstrap.cc:654] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:17.147161 10665 tablet_bootstrap.cc:492] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: No bootstrap required, opened a new log
I20260929 02:52:17.147328 10665 ts_tablet_manager.cc:1397] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:52:17.148012 10665 raft_consensus.cc:348] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.148229 10665 raft_consensus.cc:374] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:17.148315 10665 raft_consensus.cc:729] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Initialized, Role: FOLLOWER
I20260929 02:52:17.148559 10665 consensus_queue.cc:260] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.149204 10665 ts_tablet_manager.cc:1428] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:52:17.152529 10707 ts_tablet_manager.cc:1918] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:17.152694 10707 log.cc:1196] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/c998ba74d0fd4701b2ee95907c916a3e
I20260929 02:52:17.153239 10707 ts_tablet_manager.cc:1939] T c998ba74d0fd4701b2ee95907c916a3e P 6b214eee827944b1b8385aaffbfbcebc: Deleting consensus metadata
I20260929 02:52:17.154793 10143 catalog_manager.cc:4841] TS 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803): tablet c998ba74d0fd4701b2ee95907c916a3e (table test-table [id=adb447ba49274ff1b781dc5af7f6762f]) successfully deleted
I20260929 02:52:17.458598 10700 raft_consensus.cc:482] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:17.458833 10700 raft_consensus.cc:504] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.459492 10700 leader_election.cc:288] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:17.460112 10458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e634805ef0b4c6db55b940d73053dca" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c108aab89944ae90f305129bfce8dc" is_pre_election: true
I20260929 02:52:17.460189 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e634805ef0b4c6db55b940d73053dca" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b" is_pre_election: true
I20260929 02:52:17.460412 10458 raft_consensus.cc:2454] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 0.
I20260929 02:52:17.460412 10592 raft_consensus.cc:2454] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 0.
I20260929 02:52:17.460855 10246 leader_election.cc:302] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:17.461135 10700 raft_consensus.cc:2781] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:17.461268 10700 raft_consensus.cc:482] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:17.461385 10700 raft_consensus.cc:3037] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:17.464461 10700 raft_consensus.cc:504] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.465142 10700 leader_election.cc:288] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 election: Requested vote from peers a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:17.465603 10458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e634805ef0b4c6db55b940d73053dca" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c108aab89944ae90f305129bfce8dc"
I20260929 02:52:17.465634 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e634805ef0b4c6db55b940d73053dca" candidate_uuid: "6b214eee827944b1b8385aaffbfbcebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b"
I20260929 02:52:17.465808 10458 raft_consensus.cc:3037] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:17.465813 10592 raft_consensus.cc:3037] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:17.468669 10458 raft_consensus.cc:2454] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 1.
I20260929 02:52:17.472250 10246 leader_election.cc:302] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:17.472304 10592 raft_consensus.cc:2454] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b214eee827944b1b8385aaffbfbcebc in term 1.
I20260929 02:52:17.472719 10700 raft_consensus.cc:2781] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:17.472986 10700 raft_consensus.cc:686] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 1 LEADER]: Becoming Leader. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Running, Role: LEADER
I20260929 02:52:17.473410 10700 consensus_queue.cc:237] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:17.493829 10158 catalog_manager.cc:5495] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1). New cstate: current_term: 1 leader_uuid: "6b214eee827944b1b8385aaffbfbcebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:17.591189 10593 raft_consensus.cc:1261] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Refusing update from remote peer 6b214eee827944b1b8385aaffbfbcebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:17.591984 10713 consensus_queue.cc:1035] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:17.592190 10458 raft_consensus.cc:1261] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Refusing update from remote peer 6b214eee827944b1b8385aaffbfbcebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:17.592623 10713 consensus_queue.cc:1035] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 02:52:17.613234 10688 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 02:52:19.201149 10158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:19.201397 10158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:19.205297 10158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Sending DeleteTablet for 3 replicas of tablet 3e634805ef0b4c6db55b940d73053dca
I20260929 02:52:19.206015 10295 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e634805ef0b4c6db55b940d73053dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:19 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:36116
I20260929 02:52:19.206094 10434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e634805ef0b4c6db55b940d73053dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:19 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:56832
I20260929 02:52:19.206506 10572 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e634805ef0b4c6db55b940d73053dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:19 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:49830
I20260929 02:52:19.206667 10737 tablet_replica.cc:330] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: stopping tablet replica
I20260929 02:52:19.206844 10737 raft_consensus.cc:2229] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:19.207186 10737 raft_consensus.cc:2258] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:19.208238 10738 tablet_replica.cc:330] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: stopping tablet replica
I20260929 02:52:19.208421 10738 raft_consensus.cc:2229] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:19.208645 10738 raft_consensus.cc:2258] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:19.208930 10739 tablet_replica.cc:330] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: stopping tablet replica
I20260929 02:52:19.209036 10737 ts_tablet_manager.cc:1905] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:19.209172 10739 raft_consensus.cc:2229] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:19.209398 10739 raft_consensus.cc:2258] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:19.209427 10738 ts_tablet_manager.cc:1905] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:19.210202 10739 ts_tablet_manager.cc:1905] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:19.214761 10738 ts_tablet_manager.cc:1918] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:52:19.214886 10738 log.cc:1196] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/3e634805ef0b4c6db55b940d73053dca
I20260929 02:52:19.215193 10737 ts_tablet_manager.cc:1918] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:52:19.215265 10738 ts_tablet_manager.cc:1939] T 3e634805ef0b4c6db55b940d73053dca P a3c108aab89944ae90f305129bfce8dc: Deleting consensus metadata
I20260929 02:52:19.215319 10737 log.cc:1196] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/3e634805ef0b4c6db55b940d73053dca
I20260929 02:52:19.215747 10737 ts_tablet_manager.cc:1939] T 3e634805ef0b4c6db55b940d73053dca P 6b214eee827944b1b8385aaffbfbcebc: Deleting consensus metadata
I20260929 02:52:19.216248 10143 catalog_manager.cc:4841] TS a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223): tablet 3e634805ef0b4c6db55b940d73053dca (table test-table [id=0faecfecaa1e408a85ec41f8c9202f59]) successfully deleted
I20260929 02:52:19.216675 10143 catalog_manager.cc:4841] TS 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803): tablet 3e634805ef0b4c6db55b940d73053dca (table test-table [id=0faecfecaa1e408a85ec41f8c9202f59]) successfully deleted
I20260929 02:52:19.216931 10739 ts_tablet_manager.cc:1918] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:52:19.217062 10739 log.cc:1196] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/3e634805ef0b4c6db55b940d73053dca
I20260929 02:52:19.217455 10739 ts_tablet_manager.cc:1939] T 3e634805ef0b4c6db55b940d73053dca P 01807f2cf15249bd8d67af3af9be7c2b: Deleting consensus metadata
I20260929 02:52:19.218372 10143 catalog_manager.cc:4841] TS 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599): tablet 3e634805ef0b4c6db55b940d73053dca (table test-table [id=0faecfecaa1e408a85ec41f8c9202f59]) successfully deleted
Sep 29 02:52:19 dist-test-slave-1dkt krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:52:19 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:19.254300 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.241339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.8.57.62:42099)
0929 02:52:19.241875 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:19.241881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:19.241949 (+ 68us) server_negotiation.cc:407] Connection header received
0929 02:52:19.242151 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:19.242155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:19.242217 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:19.242303 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:19.243571 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.244423 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.245704 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.246102 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.246212 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.249916 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:19.249922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:19.249925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:19.249964 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:19.252256 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:19.252816 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:19.252819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:19.252822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:19.252930 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:19.253195 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:19.253197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:19.253199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:19.253441 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:19.253630 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:19.253902 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 02:52:19.254058 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":252,"threads_started":1}
I20260929 02:52:19.257654 10158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:19.258219 10158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:19.264246 10295 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab33b249887420081fb28073e063fb4 (DEFAULT_TABLE table=test-table [id=06140bb290a441b3bd82d1f9ef48c421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:19.264604 10434 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab33b249887420081fb28073e063fb4 (DEFAULT_TABLE table=test-table [id=06140bb290a441b3bd82d1f9ef48c421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:19.264753 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab33b249887420081fb28073e063fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:19.264969 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab33b249887420081fb28073e063fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:19.265152 10572 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab33b249887420081fb28073e063fb4 (DEFAULT_TABLE table=test-table [id=06140bb290a441b3bd82d1f9ef48c421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:19.265558 10572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab33b249887420081fb28073e063fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:19.270879 10748 tablet_bootstrap.cc:492] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Bootstrap starting.
I20260929 02:52:19.271530 10749 tablet_bootstrap.cc:492] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Bootstrap starting.
I20260929 02:52:19.272843 10748 tablet_bootstrap.cc:654] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:19.273265 10749 tablet_bootstrap.cc:654] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:19.275466 10749 tablet_bootstrap.cc:492] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: No bootstrap required, opened a new log
I20260929 02:52:19.275583 10749 ts_tablet_manager.cc:1397] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:52:19.275774 10750 tablet_bootstrap.cc:492] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Bootstrap starting.
I20260929 02:52:19.276227 10748 tablet_bootstrap.cc:492] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: No bootstrap required, opened a new log
I20260929 02:52:19.276365 10748 ts_tablet_manager.cc:1397] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 02:52:19.276396 10749 raft_consensus.cc:348] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.276626 10749 raft_consensus.cc:374] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:19.276717 10749 raft_consensus.cc:729] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Initialized, Role: FOLLOWER
I20260929 02:52:19.276945 10749 consensus_queue.cc:260] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.277256 10750 tablet_bootstrap.cc:654] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:19.277112 10748 raft_consensus.cc:348] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.277333 10748 raft_consensus.cc:374] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:19.277413 10748 raft_consensus.cc:729] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Initialized, Role: FOLLOWER
I20260929 02:52:19.277626 10748 consensus_queue.cc:260] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.278318 10749 ts_tablet_manager.cc:1428] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 02:52:19.278898 10748 ts_tablet_manager.cc:1428] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 02:52:19.280949 10750 tablet_bootstrap.cc:492] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: No bootstrap required, opened a new log
I20260929 02:52:19.281090 10750 ts_tablet_manager.cc:1397] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 02:52:19.281735 10750 raft_consensus.cc:348] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.281927 10750 raft_consensus.cc:374] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:19.282024 10750 raft_consensus.cc:729] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01807f2cf15249bd8d67af3af9be7c2b, State: Initialized, Role: FOLLOWER
I20260929 02:52:19.282258 10750 consensus_queue.cc:260] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.283034 10750 ts_tablet_manager.cc:1428] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:52:19.593048 10753 raft_consensus.cc:482] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:19.593273 10753 raft_consensus.cc:504] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.594725 10753 leader_election.cc:288] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:19.602104 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.595122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:39803 (local address 127.8.57.2:41105)
0929 02:52:19.595947 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:19.595964 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:19.596103 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:19.596554 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:19.596559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:19.596591 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:19.596932 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.596942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.598343 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.598349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.599857 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.599895 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.601183 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.601190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.601377 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.601423 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 02:52:19.601522 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 02:52:19.601674 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":6,"thread_start_us":533,"threads_started":1}
I20260929 02:52:19.602152 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.595122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41105 (local address 127.8.57.1:39803)
0929 02:52:19.595675 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:19.595679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:19.596044 (+ 365us) server_negotiation.cc:407] Connection header received
0929 02:52:19.596250 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:19.596256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:19.596334 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:19.597091 (+ 757us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:19.597327 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.598201 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.600057 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.600991 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.601086 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.601186 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:19.601552 (+ 366us) server_negotiation.cc:299] Negotiation successful
0929 02:52:19.601704 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":301,"threads_started":1}
I20260929 02:52:19.602485 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.596273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:51601 (local address 127.8.57.3:44599)
0929 02:52:19.596888 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:19.596893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:19.596925 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:52:19.597023 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:19.597027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:19.597084 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:19.597174 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:19.597703 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.598567 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.600404 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.601272 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.601385 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.601621 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:19.601898 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 02:52:19.602038 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":329,"threads_started":1}
I20260929 02:52:19.602725 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.596117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:44599 (local address 127.8.57.2:51601)
0929 02:52:19.596739 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:19.596757 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:19.596872 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:19.597222 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:19.597226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:19.597249 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:19.597544 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.597551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.598705 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.598709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.600276 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.600291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.601474 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.601481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.601731 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.601764 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:52:19.601861 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:52:19.601983 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":2,"thread_start_us":253,"threads_started":1}
I20260929 02:52:19.602995 10315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab33b249887420081fb28073e063fb4" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b214eee827944b1b8385aaffbfbcebc" is_pre_election: true
I20260929 02:52:19.603346 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab33b249887420081fb28073e063fb4" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b" is_pre_election: true
I20260929 02:52:19.603430 10315 raft_consensus.cc:2454] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 0.
I20260929 02:52:19.603631 10593 raft_consensus.cc:2454] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 0.
I20260929 02:52:19.604055 10386 leader_election.cc:302] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:19.604411 10753 raft_consensus.cc:2781] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:19.604535 10753 raft_consensus.cc:482] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:19.604630 10753 raft_consensus.cc:3037] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:19.606841 10753 raft_consensus.cc:504] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.607419 10753 leader_election.cc:288] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 election: Requested vote from peers 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:19.607898 10315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab33b249887420081fb28073e063fb4" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b214eee827944b1b8385aaffbfbcebc"
I20260929 02:52:19.607910 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab33b249887420081fb28073e063fb4" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b"
I20260929 02:52:19.608103 10315 raft_consensus.cc:3037] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:19.608105 10593 raft_consensus.cc:3037] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:19.610055 10593 raft_consensus.cc:2454] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 1.
I20260929 02:52:19.610145 10315 raft_consensus.cc:2454] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 1.
I20260929 02:52:19.610421 10386 leader_election.cc:302] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01807f2cf15249bd8d67af3af9be7c2b, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:19.610698 10753 raft_consensus.cc:2781] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:19.611161 10753 raft_consensus.cc:686] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 1 LEADER]: Becoming Leader. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Running, Role: LEADER
I20260929 02:52:19.611536 10753 consensus_queue.cc:237] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.614619 10158 catalog_manager.cc:5495] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c108aab89944ae90f305129bfce8dc (127.8.57.2). New cstate: current_term: 1 leader_uuid: "a3c108aab89944ae90f305129bfce8dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:19.646338 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.640655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.8.57.2:33223)
0929 02:52:19.641223 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:19.641228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:19.641251 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:52:19.641332 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:19.641335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:19.641417 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:19.641563 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:19.641901 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.642715 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.644161 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.644567 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.644656 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.645105 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:19.645676 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:19.645781 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:19.645928 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 02:52:19.645993 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":326,"threads_started":1}
I20260929 02:52:19.660687 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.655746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60477 (local address 127.8.57.1:39803)
0929 02:52:19.655975 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:19.655982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:19.656373 (+ 391us) server_negotiation.cc:407] Connection header received
0929 02:52:19.656542 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:19.656545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:19.656611 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:19.656675 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:19.657101 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.657786 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.659177 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.659573 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.659686 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.660125 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:19.660221 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:19.660283 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:19.660403 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 02:52:19.660481 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:52:19.660765 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.655678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:39803 (local address 127.8.57.2:60477)
0929 02:52:19.656285 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:19.656301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:19.656413 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:19.656713 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:19.656716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:19.656725 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:52:19.656970 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.656977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.657949 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.657954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.659040 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.659053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.659743 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.659748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.659934 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.659996 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:52:19.660308 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:52:19.660317 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:52:19.660386 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 02:52:19.660494 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":335,"threads_started":1}
I20260929 02:52:19.668807 10593 raft_consensus.cc:1261] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Refusing update from remote peer a3c108aab89944ae90f305129bfce8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:19.669099 10315 raft_consensus.cc:1261] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Refusing update from remote peer a3c108aab89944ae90f305129bfce8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:19.669396 10760 consensus_queue.cc:1035] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:19.669727 10753 consensus_queue.cc:1035] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:19.696492 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.691251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:49585 (local address 127.8.57.2:33223)
0929 02:52:19.691417 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:19.691424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:19.691818 (+ 394us) server_negotiation.cc:407] Connection header received
0929 02:52:19.692042 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:19.692047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:19.692125 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:19.692212 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:19.692592 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.693247 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.694724 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.695282 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.695390 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.695799 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:19.695930 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:19.696012 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:19.696139 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 02:52:19.696213 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 02:52:19.696573 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:19.691179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:33223 (local address 127.8.57.1:49585)
0929 02:52:19.691734 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:19.691753 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:19.691907 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:19.692229 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:19.692232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:19.692241 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:52:19.692489 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.692497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.693356 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.693360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.694600 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:19.694615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:19.695461 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:19.695476 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:19.695609 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:19.695659 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:52:19.696036 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:52:19.696048 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 02:52:19.696111 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 02:52:19.696244 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":243,"threads_started":1}
I20260929 02:52:19.733384 10158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:19.733641 10158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:19.736343 10158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Sending DeleteTablet for 3 replicas of tablet 0ab33b249887420081fb28073e063fb4
I20260929 02:52:19.737046 10295 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab33b249887420081fb28073e063fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:19 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:36116
I20260929 02:52:19.737193 10434 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab33b249887420081fb28073e063fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:19 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:56832
I20260929 02:52:19.737818 10572 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab33b249887420081fb28073e063fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:19 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:49830
I20260929 02:52:19.738142 10158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:19.738816 10158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:19.741068 10776 tablet_replica.cc:330] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: stopping tablet replica
I20260929 02:52:19.741248 10775 tablet_replica.cc:330] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: stopping tablet replica
I20260929 02:52:19.741261 10776 raft_consensus.cc:2229] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:19.741516 10775 raft_consensus.cc:2229] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:19.741868 10775 raft_consensus.cc:2258] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:19.741870 10776 raft_consensus.cc:2258] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:19.742934 10775 ts_tablet_manager.cc:1905] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:19.742944 10776 ts_tablet_manager.cc:1905] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:19.746748 10573 tablet_service.cc:1467] Processing CreateTablet for tablet f4c4573b36314388bb9bbff3e15f73e1 (DEFAULT_TABLE table=test-table [id=95bba366fbc740b69d942029ec12faa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:19.747087 10295 tablet_service.cc:1467] Processing CreateTablet for tablet f4c4573b36314388bb9bbff3e15f73e1 (DEFAULT_TABLE table=test-table [id=95bba366fbc740b69d942029ec12faa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:19.747192 10573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c4573b36314388bb9bbff3e15f73e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:19.747521 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c4573b36314388bb9bbff3e15f73e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:19.749863 10434 tablet_service.cc:1467] Processing CreateTablet for tablet f4c4573b36314388bb9bbff3e15f73e1 (DEFAULT_TABLE table=test-table [id=95bba366fbc740b69d942029ec12faa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:19.750357 10775 ts_tablet_manager.cc:1918] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:19.750530 10775 log.cc:1196] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/0ab33b249887420081fb28073e063fb4
I20260929 02:52:19.751044 10775 ts_tablet_manager.cc:1939] T 0ab33b249887420081fb28073e063fb4 P 6b214eee827944b1b8385aaffbfbcebc: Deleting consensus metadata
I20260929 02:52:19.752086 10143 catalog_manager.cc:4841] TS 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803): tablet 0ab33b249887420081fb28073e063fb4 (table test-table [id=06140bb290a441b3bd82d1f9ef48c421]) successfully deleted
I20260929 02:52:19.753134 10750 tablet_bootstrap.cc:492] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Bootstrap starting.
I20260929 02:52:19.754256 10777 tablet_replica.cc:330] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: stopping tablet replica
I20260929 02:52:19.754390 10749 tablet_bootstrap.cc:492] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Bootstrap starting.
I20260929 02:52:19.754446 10777 raft_consensus.cc:2229] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:19.754657 10777 raft_consensus.cc:2258] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:19.754859 10750 tablet_bootstrap.cc:654] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:19.754927 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c4573b36314388bb9bbff3e15f73e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:19.755942 10749 tablet_bootstrap.cc:654] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:19.760840 10777 ts_tablet_manager.cc:1905] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:19.761441 10748 tablet_bootstrap.cc:492] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Bootstrap starting.
I20260929 02:52:19.763089 10748 tablet_bootstrap.cc:654] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:19.767429 10777 ts_tablet_manager.cc:1918] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:19.767567 10777 log.cc:1196] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/0ab33b249887420081fb28073e063fb4
I20260929 02:52:19.769337 10777 ts_tablet_manager.cc:1939] T 0ab33b249887420081fb28073e063fb4 P 01807f2cf15249bd8d67af3af9be7c2b: Deleting consensus metadata
I20260929 02:52:19.770473 10750 tablet_bootstrap.cc:492] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: No bootstrap required, opened a new log
I20260929 02:52:19.770625 10750 ts_tablet_manager.cc:1397] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.006s
I20260929 02:52:19.771183 10750 raft_consensus.cc:348] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.771373 10750 raft_consensus.cc:374] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:19.771447 10750 raft_consensus.cc:729] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01807f2cf15249bd8d67af3af9be7c2b, State: Initialized, Role: FOLLOWER
I20260929 02:52:19.771678 10750 consensus_queue.cc:260] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.772104 10750 ts_tablet_manager.cc:1428] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260929 02:52:19.772696 10143 catalog_manager.cc:4841] TS 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599): tablet 0ab33b249887420081fb28073e063fb4 (table test-table [id=06140bb290a441b3bd82d1f9ef48c421]) successfully deleted
I20260929 02:52:19.774546 10749 tablet_bootstrap.cc:492] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: No bootstrap required, opened a new log
I20260929 02:52:19.774893 10749 ts_tablet_manager.cc:1397] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.009s
I20260929 02:52:19.775761 10749 raft_consensus.cc:348] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.776031 10749 raft_consensus.cc:374] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:19.776116 10749 raft_consensus.cc:729] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b214eee827944b1b8385aaffbfbcebc, State: Initialized, Role: FOLLOWER
I20260929 02:52:19.776376 10749 consensus_queue.cc:260] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.776930 10749 ts_tablet_manager.cc:1428] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:52:19.778543 10776 ts_tablet_manager.cc:1918] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:52:19.779626 10748 tablet_bootstrap.cc:492] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: No bootstrap required, opened a new log
I20260929 02:52:19.779754 10776 log.cc:1196] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/0ab33b249887420081fb28073e063fb4
I20260929 02:52:19.779966 10748 ts_tablet_manager.cc:1397] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.007s
I20260929 02:52:19.780280 10776 ts_tablet_manager.cc:1939] T 0ab33b249887420081fb28073e063fb4 P a3c108aab89944ae90f305129bfce8dc: Deleting consensus metadata
I20260929 02:52:19.780778 10748 raft_consensus.cc:348] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.780977 10748 raft_consensus.cc:374] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:19.781060 10748 raft_consensus.cc:729] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Initialized, Role: FOLLOWER
I20260929 02:52:19.781297 10143 catalog_manager.cc:4841] TS a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223): tablet 0ab33b249887420081fb28073e063fb4 (table test-table [id=06140bb290a441b3bd82d1f9ef48c421]) successfully deleted
I20260929 02:52:19.781302 10748 consensus_queue.cc:260] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.781786 10748 ts_tablet_manager.cc:1428] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 02:52:19.906316 10753 raft_consensus.cc:482] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:19.906545 10753 raft_consensus.cc:504] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.907267 10753 leader_election.cc:288] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:19.907850 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c4573b36314388bb9bbff3e15f73e1" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b" is_pre_election: true
I20260929 02:52:19.907974 10315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c4573b36314388bb9bbff3e15f73e1" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b214eee827944b1b8385aaffbfbcebc" is_pre_election: true
I20260929 02:52:19.908170 10593 raft_consensus.cc:2454] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 0.
I20260929 02:52:19.908253 10315 raft_consensus.cc:2454] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 0.
I20260929 02:52:19.908687 10386 leader_election.cc:302] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01807f2cf15249bd8d67af3af9be7c2b, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:19.908993 10753 raft_consensus.cc:2781] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:19.909128 10753 raft_consensus.cc:482] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:19.909219 10753 raft_consensus.cc:3037] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:19.912292 10753 raft_consensus.cc:504] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.912931 10753 leader_election.cc:288] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 election: Requested vote from peers 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803), 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599)
I20260929 02:52:19.913391 10315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c4573b36314388bb9bbff3e15f73e1" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b214eee827944b1b8385aaffbfbcebc"
I20260929 02:52:19.913492 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c4573b36314388bb9bbff3e15f73e1" candidate_uuid: "a3c108aab89944ae90f305129bfce8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01807f2cf15249bd8d67af3af9be7c2b"
I20260929 02:52:19.913606 10315 raft_consensus.cc:3037] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:19.913739 10593 raft_consensus.cc:3037] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:19.916503 10593 raft_consensus.cc:2454] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 1.
I20260929 02:52:19.916500 10315 raft_consensus.cc:2454] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c108aab89944ae90f305129bfce8dc in term 1.
I20260929 02:52:19.916913 10386 leader_election.cc:302] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b214eee827944b1b8385aaffbfbcebc, a3c108aab89944ae90f305129bfce8dc; no voters:
I20260929 02:52:19.917179 10753 raft_consensus.cc:2781] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:19.917347 10753 raft_consensus.cc:686] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 1 LEADER]: Becoming Leader. State: Replica: a3c108aab89944ae90f305129bfce8dc, State: Running, Role: LEADER
I20260929 02:52:19.917855 10753 consensus_queue.cc:237] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } }
I20260929 02:52:19.921293 10158 catalog_manager.cc:5495] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c108aab89944ae90f305129bfce8dc (127.8.57.2). New cstate: current_term: 1 leader_uuid: "a3c108aab89944ae90f305129bfce8dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c108aab89944ae90f305129bfce8dc" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 33223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:19.963402 10592 raft_consensus.cc:1261] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Refusing update from remote peer a3c108aab89944ae90f305129bfce8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:19.964206 10753 consensus_queue.cc:1035] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01807f2cf15249bd8d67af3af9be7c2b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:19.964524 10315 raft_consensus.cc:1261] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Refusing update from remote peer a3c108aab89944ae90f305129bfce8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:19.965298 10753 consensus_queue.cc:1035] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b214eee827944b1b8385aaffbfbcebc" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 39803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260929 02:52:20.000108 10722 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 02:52:21.588402 10158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:21.588619 10158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:21.592199 10158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9523d6c1e2fa49bda91b3b60f289073f: Sending DeleteTablet for 3 replicas of tablet f4c4573b36314388bb9bbff3e15f73e1
I20260929 02:52:21.592841 10295 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c4573b36314388bb9bbff3e15f73e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:21 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:36116
I20260929 02:52:21.592912 10434 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c4573b36314388bb9bbff3e15f73e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:21 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:56832
I20260929 02:52:21.592990 10573 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c4573b36314388bb9bbff3e15f73e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:21 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:49830
I20260929 02:52:21.594521 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10229
I20260929 02:52:21.605499 10805 tablet_replica.cc:330] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: stopping tablet replica
I20260929 02:52:21.605829 10805 raft_consensus.cc:2229] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:21.606210 10805 raft_consensus.cc:2258] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:21.607436 10805 ts_tablet_manager.cc:1905] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:21.609567 10806 tablet_replica.cc:330] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: stopping tablet replica
I20260929 02:52:21.609901 10806 raft_consensus.cc:2229] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:21.610328 10806 raft_consensus.cc:2258] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:21.611307 10806 ts_tablet_manager.cc:1905] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:21.614884 10805 ts_tablet_manager.cc:1918] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:52:21.615386 10807 tablet_replica.cc:330] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: stopping tablet replica
I20260929 02:52:21.615581 10807 raft_consensus.cc:2229] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:21.615797 10807 raft_consensus.cc:2258] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:21.616729 10807 ts_tablet_manager.cc:1905] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:21.617926 10806 ts_tablet_manager.cc:1918] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:52:21.618072 10806 log.cc:1196] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/f4c4573b36314388bb9bbff3e15f73e1
I20260929 02:52:21.618454 10806 ts_tablet_manager.cc:1939] T f4c4573b36314388bb9bbff3e15f73e1 P a3c108aab89944ae90f305129bfce8dc: Deleting consensus metadata
I20260929 02:52:21.619626 10143 catalog_manager.cc:4841] TS a3c108aab89944ae90f305129bfce8dc (127.8.57.2:33223): tablet f4c4573b36314388bb9bbff3e15f73e1 (table test-table [id=95bba366fbc740b69d942029ec12faa6]) successfully deleted
I20260929 02:52:21.616056 10805 log.cc:1196] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/f4c4573b36314388bb9bbff3e15f73e1
I20260929 02:52:21.620374 10805 ts_tablet_manager.cc:1939] T f4c4573b36314388bb9bbff3e15f73e1 P 6b214eee827944b1b8385aaffbfbcebc: Deleting consensus metadata
I20260929 02:52:21.621518 10143 catalog_manager.cc:4841] TS 6b214eee827944b1b8385aaffbfbcebc (127.8.57.1:39803): tablet f4c4573b36314388bb9bbff3e15f73e1 (table test-table [id=95bba366fbc740b69d942029ec12faa6]) successfully deleted
I20260929 02:52:21.621902 10807 ts_tablet_manager.cc:1918] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:52:21.622004 10807 log.cc:1196] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/f4c4573b36314388bb9bbff3e15f73e1
I20260929 02:52:21.622390 10807 ts_tablet_manager.cc:1939] T f4c4573b36314388bb9bbff3e15f73e1 P 01807f2cf15249bd8d67af3af9be7c2b: Deleting consensus metadata
I20260929 02:52:21.623306 10143 catalog_manager.cc:4841] TS 01807f2cf15249bd8d67af3af9be7c2b (127.8.57.3:44599): tablet f4c4573b36314388bb9bbff3e15f73e1 (table test-table [id=95bba366fbc740b69d942029ec12faa6]) successfully deleted
Sep 29 02:52:21 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:21.633731 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:21.596131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.8.57.1:39803)
0929 02:52:21.619648 (+ 23517us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:21.619655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:21.620176 (+ 521us) server_negotiation.cc:407] Connection header received
0929 02:52:21.620344 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:21.620349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:21.620438 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:21.620570 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:21.622160 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:21.623204 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:21.624495 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:21.624907 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:21.625029 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:21.628614 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:21.628626 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:21.628642 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:21.628672 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:21.631129 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:21.631756 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:21.631763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:21.631766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:21.631956 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:21.632300 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:21.632305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:21.632308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:21.632651 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:21.632925 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:21.633195 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 02:52:21.633399 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23376,"thread_start_us":23265,"threads_started":1}
W20260929 02:52:21.634312 10360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41144
W20260929 02:52:21.634848 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10229: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:21.635035 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 10229
I20260929 02:52:21.655189 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10369
Sep 29 02:52:21 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:21.669131 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:21.655867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.8.57.2:33223)
0929 02:52:21.656396 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:21.656400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:21.656415 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:52:21.656467 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:21.656471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:21.656537 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:21.656624 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:21.657756 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:21.658505 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:21.659651 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:21.660120 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:21.660219 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:21.664015 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:21.664028 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:21.664042 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:21.664078 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:21.666529 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:21.667159 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:21.667166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:21.667168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:21.667296 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:21.667651 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:21.667655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:21.667657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:21.668076 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:21.668310 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:21.668629 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 02:52:21.668795 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":286,"threads_started":1}
W20260929 02:52:21.669764 10499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58594
W20260929 02:52:21.670333 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:21.670477 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 10369
I20260929 02:52:21.686631 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10508
Sep 29 02:52:21 dist-test-slave-1dkt krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650339, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:21.700918 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:21.687352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59962 (local address 127.8.57.3:44599)
0929 02:52:21.687845 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:21.687850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:21.687864 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:52:21.688009 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:21.688013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:21.688075 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:21.688169 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:21.689361 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:21.690064 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:21.691220 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:21.691660 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:21.691766 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:21.695746 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:21.695760 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:21.695775 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:21.695813 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:21.698363 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:21.698965 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:21.698972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:21.698974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:21.699113 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:21.699442 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:21.699446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:21.699448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:21.699806 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:21.700103 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:21.700361 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 02:52:21.700577 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":246,"threads_started":1}
W20260929 02:52:21.701553 10638 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59962
W20260929 02:52:21.702159 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10508: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:21.702306 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 10508
I20260929 02:52:21.715466 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10126
I20260929 02:52:21.716629 10198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:21.830596 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 10126
2026-09-29T02:52:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8518 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:21 dist-test-slave-1dkt krb5kdc[10815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:21 dist-test-slave-1dkt krb5kdc[10815](info): set up 2 sockets
Sep 29 02:52:21 dist-test-slave-1dkt krb5kdc[10815](info): commencing operation
krb5kdc: starting...
W20260929 02:52:22.828032 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:22 dist-test-slave-1dkt krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:22.993040 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40265
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:23.296762 10828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:23.297071 10828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:23.297210 10828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:23.304345 10828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:23.304479 10828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:23.304555 10828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:23.304633 10828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:23.304693 10828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:23.314514 10828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:23.315791 10828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:23.317076 10828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:23.328053 10833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:23.328024 10836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:23.328024 10834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:23.329003 10828 server_base.cc:1034] running on GCE node
I20260929 02:52:23.329720 10828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:23.331010 10828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:23.332206 10828 hybrid_clock.cc:648] HybridClock initialized: now 1790650343332162 us; error 29 us; skew 500 ppm
I20260929 02:52:23.332537 10828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:23 dist-test-slave-1dkt krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650343, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:23.335368 10828 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:23.336761 10828 webserver.cc:460] Webserver started at http://127.8.57.62:38125/ using document root <none> and password file <none>
I20260929 02:52:23.337392 10828 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:23.337503 10828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:23.337795 10828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:23.340677 10828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "7538e5ace21e4a55a0a8abb4a5771c30"
format_stamp: "Formatted at 2026-09-29 02:52:23 on dist-test-slave-1dkt"
server_key: "4b48b8357f0ed54a4f458224811b6ac9"
server_key_iv: "723f2e61009445fcd8d8df81da385ee5"
server_key_version: "encryptionkey@0"
I20260929 02:52:23.341406 10828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "7538e5ace21e4a55a0a8abb4a5771c30"
format_stamp: "Formatted at 2026-09-29 02:52:23 on dist-test-slave-1dkt"
server_key: "4b48b8357f0ed54a4f458224811b6ac9"
server_key_iv: "723f2e61009445fcd8d8df81da385ee5"
server_key_version: "encryptionkey@0"
I20260929 02:52:23.346246 10828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:52:23.349390 10843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:23.350617 10828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:52:23.350770 10828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "7538e5ace21e4a55a0a8abb4a5771c30"
format_stamp: "Formatted at 2026-09-29 02:52:23 on dist-test-slave-1dkt"
server_key: "4b48b8357f0ed54a4f458224811b6ac9"
server_key_iv: "723f2e61009445fcd8d8df81da385ee5"
server_key_version: "encryptionkey@0"
I20260929 02:52:23.350935 10828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:23.366216 10828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:23.370055 10828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:23.370309 10828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:23.387044 10828 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40265
I20260929 02:52:23.387095 10894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40265 every 8 connection(s)
I20260929 02:52:23.388373 10828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:23.391981 10895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:23.397640 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 10828
I20260929 02:52:23.397872 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:23.398448 8420 external_mini_cluster.cc:1428] Setting key 6162921f5524ff60656fa80eab3140e3
I20260929 02:52:23.402407 10895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30: Bootstrap starting.
I20260929 02:52:23.406066 10895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:23.407294 10895 log.cc:824] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:23.410291 10895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30: No bootstrap required, opened a new log
Sep 29 02:52:23 dist-test-slave-1dkt krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:23.419010 10895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40265 } }
I20260929 02:52:23.419437 10895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:23.419553 10895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7538e5ace21e4a55a0a8abb4a5771c30, State: Initialized, Role: FOLLOWER
I20260929 02:52:23.420253 10895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40265 } }
I20260929 02:52:23.420514 10895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:23.420648 10895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:23.420820 10895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:23.421499 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:23.400338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.8.57.62:40265)
0929 02:52:23.401077 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:23.401092 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:23.401141 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:52:23.403350 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:23.403378 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:23.403852 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:23.404321 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:23.405550 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:23.407485 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:23.409013 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:23.409971 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:23.410127 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:23.413559 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:23.413569 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:23.413587 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:23.413626 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:23.417366 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:23.417976 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:23.417984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:23.417993 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:23.418192 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:23.418490 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:23.418494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:23.418497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:23.419152 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:23.419420 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:23.419762 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 02:52:23.420189 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":298,"threads_started":1}
I20260929 02:52:23.423238 10895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40265 } }
I20260929 02:52:23.423696 10895 leader_election.cc:302] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7538e5ace21e4a55a0a8abb4a5771c30; no voters:
I20260929 02:52:23.424288 10895 leader_election.cc:288] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:23.424382 10900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:23.424945 10900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [term 1 LEADER]: Becoming Leader. State: Replica: 7538e5ace21e4a55a0a8abb4a5771c30, State: Running, Role: LEADER
I20260929 02:52:23.425473 10900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40265 } }
I20260929 02:52:23.426294 10895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:23.428333 10902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7538e5ace21e4a55a0a8abb4a5771c30. Latest consensus state: current_term: 1 leader_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40265 } } }
I20260929 02:52:23.428340 10901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7538e5ace21e4a55a0a8abb4a5771c30" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40265 } } }
I20260929 02:52:23.428762 10902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:23.428761 10901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:23.430059 10905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:23.434609 10905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:23.442371 10905 catalog_manager.cc:1347] Generated new cluster ID: ecd54148737e4ed08f8a6dc702f1bc8c
I20260929 02:52:23.442476 10905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:23.459431 10905 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:23.461244 10905 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:23.477943 10905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7538e5ace21e4a55a0a8abb4a5771c30: Generated new TSK 0
I20260929 02:52:23.478741 10905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:23.519173 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40265
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:23.822371 10921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:23.822657 10921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:23.822790 10921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:23.829948 10921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:23.830060 10921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:23.830206 10921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:23.839689 10921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:23.840896 10921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:23.842126 10921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:23.854069 10926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:23.854019 10927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:23.854101 10929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:23.854812 10921 server_base.cc:1034] running on GCE node
I20260929 02:52:23.855350 10921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:23.856222 10921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:23.857396 10921 hybrid_clock.cc:648] HybridClock initialized: now 1790650343857347 us; error 43 us; skew 500 ppm
I20260929 02:52:23.857650 10921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:23 dist-test-slave-1dkt krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650343, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:23.860466 10921 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:23.861747 10921 webserver.cc:460] Webserver started at http://127.8.57.1:37025/ using document root <none> and password file <none>
I20260929 02:52:23.862342 10921 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:23.862445 10921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:23.862697 10921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:23.865553 10921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "19e32b91d13549a391e3a5ba01e76931"
format_stamp: "Formatted at 2026-09-29 02:52:23 on dist-test-slave-1dkt"
server_key: "b0b8a64360327e2f3afa3f2f3d249928"
server_key_iv: "94200b35c891169e02177c601be00bbe"
server_key_version: "encryptionkey@0"
I20260929 02:52:23.866247 10921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "19e32b91d13549a391e3a5ba01e76931"
format_stamp: "Formatted at 2026-09-29 02:52:23 on dist-test-slave-1dkt"
server_key: "b0b8a64360327e2f3afa3f2f3d249928"
server_key_iv: "94200b35c891169e02177c601be00bbe"
server_key_version: "encryptionkey@0"
I20260929 02:52:23.870929 10921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:52:23.874011 10936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:23.875245 10921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:52:23.875362 10921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "19e32b91d13549a391e3a5ba01e76931"
format_stamp: "Formatted at 2026-09-29 02:52:23 on dist-test-slave-1dkt"
server_key: "b0b8a64360327e2f3afa3f2f3d249928"
server_key_iv: "94200b35c891169e02177c601be00bbe"
server_key_version: "encryptionkey@0"
I20260929 02:52:23.875506 10921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:23.889714 10921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:23.893759 10921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:23.893997 10921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:23.894810 10921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:23.896090 10921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:23.896160 10921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:23.896251 10921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:23.896296 10921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:23.924268 10921 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:45225
I20260929 02:52:23.924309 11048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:45225 every 8 connection(s)
I20260929 02:52:23.925443 10921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:23.925904 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 10921
I20260929 02:52:23.926124 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:23.926903 8420 external_mini_cluster.cc:1428] Setting key 9a928c694a18540510d01505170eb302
Sep 29 02:52:23 dist-test-slave-1dkt krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650343, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:23.944795 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:23.928029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:49127 (local address 127.8.57.62:40265)
0929 02:52:23.928475 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:23.928480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:23.929383 (+ 903us) server_negotiation.cc:407] Connection header received
0929 02:52:23.931240 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:23.931245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:23.931322 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:23.931413 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:23.932947 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:23.933981 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:23.935476 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:23.936117 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:23.936251 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:23.939598 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:23.939603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:23.939607 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:23.939638 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:23.941971 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:23.942798 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:23.942802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:23.942804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:23.942919 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:23.943250 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:23.943253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:23.943255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:23.943499 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:23.943656 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:23.944296 (+ 640us) server_negotiation.cc:299] Negotiation successful
0929 02:52:23.944498 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":222,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:52:23.945662 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:23.928593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40265 (local address 127.8.57.1:49127)
0929 02:52:23.929258 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:23.929293 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:23.930994 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:23.931650 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:23.931658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:23.932261 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:23.932795 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:23.932812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:23.934127 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:23.934131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:23.935326 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:23.935337 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:23.936324 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:23.936330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:23.936532 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:23.937131 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:23.937152 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:23.939391 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:23.942110 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:23.942119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:23.942136 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:23.942699 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:23.942999 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:23.943003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:23.943005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:23.943177 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:23.943627 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:23.943634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:23.943970 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 02:52:23.944190 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 02:52:23.944560 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":276,"threads_started":1}
I20260929 02:52:23.947197 11049 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40265
I20260929 02:52:23.947464 11049 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:23.948171 11049 heartbeater.cc:511] Master 127.8.57.62:40265 requested a full tablet report, sending...
I20260929 02:52:23.950317 10860 ts_manager.cc:194] Registered new tserver with Master: 19e32b91d13549a391e3a5ba01e76931 (127.8.57.1:45225)
I20260929 02:52:23.951707 10860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:49127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:23.966111 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40265
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:24.272362 11055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:24.272645 11055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:24.272737 11055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:24.280108 11055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:24.280221 11055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:24.280333 11055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:24.289953 11055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:24.291154 11055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:24.292413 11055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:24.305055 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:24.305055 11061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:24.305202 11060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:24.305627 11055 server_base.cc:1034] running on GCE node
I20260929 02:52:24.306298 11055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:24.307219 11055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:24.308404 11055 hybrid_clock.cc:648] HybridClock initialized: now 1790650344308357 us; error 38 us; skew 500 ppm
I20260929 02:52:24.308722 11055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:24 dist-test-slave-1dkt krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650344, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:24.311646 11055 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:24.313072 11055 webserver.cc:460] Webserver started at http://127.8.57.2:43141/ using document root <none> and password file <none>
I20260929 02:52:24.313728 11055 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:24.313838 11055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:24.314118 11055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:24.316977 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "4e0450dc886b4a2c9be69b679b799c05"
format_stamp: "Formatted at 2026-09-29 02:52:24 on dist-test-slave-1dkt"
server_key: "5c4b0fcc13d6c3b0dde0fa7331c8099a"
server_key_iv: "a9f3dfa7addc0b2da24ce2ec4e601efb"
server_key_version: "encryptionkey@0"
I20260929 02:52:24.317687 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "4e0450dc886b4a2c9be69b679b799c05"
format_stamp: "Formatted at 2026-09-29 02:52:24 on dist-test-slave-1dkt"
server_key: "5c4b0fcc13d6c3b0dde0fa7331c8099a"
server_key_iv: "a9f3dfa7addc0b2da24ce2ec4e601efb"
server_key_version: "encryptionkey@0"
I20260929 02:52:24.322445 11055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:52:24.325747 11070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:24.327014 11055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:24.327154 11055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "4e0450dc886b4a2c9be69b679b799c05"
format_stamp: "Formatted at 2026-09-29 02:52:24 on dist-test-slave-1dkt"
server_key: "5c4b0fcc13d6c3b0dde0fa7331c8099a"
server_key_iv: "a9f3dfa7addc0b2da24ce2ec4e601efb"
server_key_version: "encryptionkey@0"
I20260929 02:52:24.327325 11055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:24.357060 11055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:24.360697 11055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:24.360942 11055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:24.361825 11055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:24.363087 11055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:24.363161 11055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:24.363279 11055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:24.363332 11055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:24.392340 11055 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:45885
I20260929 02:52:24.392407 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:45885 every 8 connection(s)
I20260929 02:52:24.393579 11055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:24.401455 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11055
I20260929 02:52:24.401624 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:24.402169 8420 external_mini_cluster.cc:1428] Setting key 766125e639fce99af7cad0591be223b0
Sep 29 02:52:24 dist-test-slave-1dkt krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650344, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:24.414505 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:24.396028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60411 (local address 127.8.57.62:40265)
0929 02:52:24.396218 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:24.396224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:24.397428 (+ 1204us) server_negotiation.cc:407] Connection header received
0929 02:52:24.399814 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:24.399818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:24.399917 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:24.400012 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:24.401865 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.402894 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.405143 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.405781 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.405924 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:24.409153 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:24.409161 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:24.409165 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:24.409203 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:24.411647 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:24.412433 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:24.412436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:24.412438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:24.412562 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:24.412971 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:24.412974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:24.412976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:24.413214 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:24.413312 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:24.414121 (+ 809us) server_negotiation.cc:299] Negotiation successful
0929 02:52:24.414284 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:52:24.416038 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:24.396489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40265 (local address 127.8.57.2:60411)
0929 02:52:24.397258 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:24.397303 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:24.399534 (+ 2231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:24.400256 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:24.400266 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:24.400973 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:24.401676 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:24.401698 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.403047 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.403053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:24.404980 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:24.404995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.405985 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.405990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:24.406201 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:24.406842 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:24.406863 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:24.408934 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:24.411773 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:24.411782 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:24.411805 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:24.412336 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:24.412656 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:24.412660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:24.412663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:24.412884 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:24.413320 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:24.413328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:24.413765 (+ 437us) client_negotiation.cc:769] Sending connection context
0929 02:52:24.414048 (+ 283us) client_negotiation.cc:240] Negotiation successful
0929 02:52:24.414578 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":275,"threads_started":1}
I20260929 02:52:24.417371 11183 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40265
I20260929 02:52:24.417752 11183 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:24.418562 11183 heartbeater.cc:511] Master 127.8.57.62:40265 requested a full tablet report, sending...
I20260929 02:52:24.419907 10860 ts_manager.cc:194] Registered new tserver with Master: 4e0450dc886b4a2c9be69b679b799c05 (127.8.57.2:45885)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
I20260929 02:52:24.420795 10860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:60411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:24.440815 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40265
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:24.742307 11188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:24.742573 11188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:24.742700 11188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:24.749686 11188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:24.749801 11188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:24.749948 11188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:24.759394 11188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:24.760622 11188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:24.761827 11188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:24.773715 11193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:24.773753 11196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:24.773730 11194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:24.774441 11188 server_base.cc:1034] running on GCE node
I20260929 02:52:24.774952 11188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:24.775800 11188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:24.776981 11188 hybrid_clock.cc:648] HybridClock initialized: now 1790650344776953 us; error 61 us; skew 500 ppm
I20260929 02:52:24.777263 11188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:24 dist-test-slave-1dkt krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650344, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:24.780211 11188 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:24.781505 11188 webserver.cc:460] Webserver started at http://127.8.57.3:34767/ using document root <none> and password file <none>
I20260929 02:52:24.782130 11188 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:24.782240 11188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:24.782517 11188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:24.785319 11188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "9e44452f12fa4c8fa595f669acaa4cc6"
format_stamp: "Formatted at 2026-09-29 02:52:24 on dist-test-slave-1dkt"
server_key: "ba2d9bd98b315c3dfbe013d9620a5a58"
server_key_iv: "8e3c828e91ddd26ba02ed8970bd051e0"
server_key_version: "encryptionkey@0"
I20260929 02:52:24.786078 11188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "9e44452f12fa4c8fa595f669acaa4cc6"
format_stamp: "Formatted at 2026-09-29 02:52:24 on dist-test-slave-1dkt"
server_key: "ba2d9bd98b315c3dfbe013d9620a5a58"
server_key_iv: "8e3c828e91ddd26ba02ed8970bd051e0"
server_key_version: "encryptionkey@0"
I20260929 02:52:24.790905 11188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:52:24.794066 11203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:24.795265 11188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 02:52:24.795408 11188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "9e44452f12fa4c8fa595f669acaa4cc6"
format_stamp: "Formatted at 2026-09-29 02:52:24 on dist-test-slave-1dkt"
server_key: "ba2d9bd98b315c3dfbe013d9620a5a58"
server_key_iv: "8e3c828e91ddd26ba02ed8970bd051e0"
server_key_version: "encryptionkey@0"
I20260929 02:52:24.795591 11188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:24.831606 11188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:24.835536 11188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:24.835815 11188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:24.836807 11188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:24.838060 11188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:24.838141 11188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:24.838256 11188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:24.838305 11188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:24.866122 11188 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:43561
I20260929 02:52:24.866173 11315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:43561 every 8 connection(s)
I20260929 02:52:24.867326 11188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:24.874715 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11188
I20260929 02:52:24.874915 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:24.875383 8420 external_mini_cluster.cc:1428] Setting key 9007b1f3a11b7617d1ca39f348207072
Sep 29 02:52:24 dist-test-slave-1dkt krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650344, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:24.885758 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:24.869697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:43259 (local address 127.8.57.62:40265)
0929 02:52:24.869865 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:24.869869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:24.870909 (+ 1040us) server_negotiation.cc:407] Connection header received
0929 02:52:24.872645 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:24.872650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:24.872720 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:24.872801 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:24.874147 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.875118 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.876891 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.877392 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.877481 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:24.880811 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:24.880816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:24.880820 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:24.880853 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:24.883101 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:24.883755 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:24.883760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:24.883762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:24.883907 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:24.884354 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:24.884357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:24.884359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:24.884623 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:24.884729 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:24.885349 (+ 620us) server_negotiation.cc:299] Negotiation successful
0929 02:52:24.885513 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:52:24.886709 11318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:24.870118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40265 (local address 127.8.57.3:43259)
0929 02:52:24.870754 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:24.870792 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:24.872410 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:24.872982 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:24.872990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:24.873486 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:24.874009 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:24.874025 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.875252 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.875255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:24.876762 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:24.876774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:24.877600 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:24.877607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:24.877873 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:24.878507 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:24.878527 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:24.880627 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:24.883229 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:24.883236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:24.883249 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:24.883653 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:24.884012 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:24.884017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:24.884021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:24.884254 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:24.884713 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:24.884718 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:24.885028 (+ 310us) client_negotiation.cc:769] Sending connection context
0929 02:52:24.885264 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 02:52:24.885650 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":254,"threads_started":1}
I20260929 02:52:24.887971 11316 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40265
I20260929 02:52:24.888276 11316 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:24.888998 11316 heartbeater.cc:511] Master 127.8.57.62:40265 requested a full tablet report, sending...
I20260929 02:52:24.890323 10860 ts_manager.cc:194] Registered new tserver with Master: 9e44452f12fa4c8fa595f669acaa4cc6 (127.8.57.3:43561)
I20260929 02:52:24.891114 10860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:43259
I20260929 02:52:24.902096 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:52:24.928188 11051 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:24.925578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.8.57.62:40265)
0929 02:52:24.925795 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:24.925802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:24.926307 (+ 505us) server_negotiation.cc:407] Connection header received
0929 02:52:24.926508 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:24.926513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:24.926580 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:24.926659 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:24.927934 (+ 1275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38702: BlockingRecv error: recv got EOF from 127.0.0.1:38702 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:52:24.931466 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10921
W20260929 02:52:24.933842 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10921: Not authorized: Client connection negotiation failed: client connection to 127.8.57.1:45225: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:52:24.934013 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 10921
I20260929 02:52:24.946269 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11055
W20260929 02:52:24.948707 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11055: Not authorized: Client connection negotiation failed: client connection to 127.8.57.2:45885: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:52:24.948861 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11055
I20260929 02:52:24.960232 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11188
W20260929 02:52:24.962900 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11188: Not authorized: Client connection negotiation failed: client connection to 127.8.57.3:43561: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:52:24.963075 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11188
I20260929 02:52:24.974314 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 10828
I20260929 02:52:24.975451 10890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:25.076345 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 10828
2026-09-29T02:52:25Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3244 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:25 dist-test-slave-1dkt krb5kdc[11333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:25 dist-test-slave-1dkt krb5kdc[11333](info): set up 2 sockets
Sep 29 02:52:25 dist-test-slave-1dkt krb5kdc[11333](info): commencing operation
krb5kdc: starting...
W20260929 02:52:26.060027 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.001s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:26 dist-test-slave-1dkt krb5kdc[11333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:26.234673 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40775
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:26.543572 11346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:26.543905 11346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:26.544054 11346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:26.551244 11346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:26.551369 11346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:26.551452 11346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:26.551538 11346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:26.551601 11346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:26.561394 11346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:26.562659 11346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:26.563978 11346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:26.575135 11351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:26.575114 11352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:26.575114 11354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:26.575994 11346 server_base.cc:1034] running on GCE node
I20260929 02:52:26.576748 11346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:26.578147 11346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:26.579339 11346 hybrid_clock.cc:648] HybridClock initialized: now 1790650346579288 us; error 43 us; skew 500 ppm
I20260929 02:52:26.579666 11346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:26 dist-test-slave-1dkt krb5kdc[11333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650346, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:26.582767 11346 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:26.584223 11346 webserver.cc:460] Webserver started at http://127.8.57.62:44801/ using document root <none> and password file <none>
I20260929 02:52:26.584877 11346 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:26.584985 11346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:26.585274 11346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:26.588246 11346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "3e547ab8ce05497da5e50c0e8cf8df9e"
format_stamp: "Formatted at 2026-09-29 02:52:26 on dist-test-slave-1dkt"
server_key: "1cf89ce0d015c4bf51dd315c2e9e89a3"
server_key_iv: "42b9d95e06a8125b240e511e199b37f6"
server_key_version: "encryptionkey@0"
I20260929 02:52:26.589039 11346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "3e547ab8ce05497da5e50c0e8cf8df9e"
format_stamp: "Formatted at 2026-09-29 02:52:26 on dist-test-slave-1dkt"
server_key: "1cf89ce0d015c4bf51dd315c2e9e89a3"
server_key_iv: "42b9d95e06a8125b240e511e199b37f6"
server_key_version: "encryptionkey@0"
I20260929 02:52:26.594122 11346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:26.597461 11361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:26.598742 11346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:52:26.598888 11346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "3e547ab8ce05497da5e50c0e8cf8df9e"
format_stamp: "Formatted at 2026-09-29 02:52:26 on dist-test-slave-1dkt"
server_key: "1cf89ce0d015c4bf51dd315c2e9e89a3"
server_key_iv: "42b9d95e06a8125b240e511e199b37f6"
server_key_version: "encryptionkey@0"
I20260929 02:52:26.599058 11346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:26.618201 11346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:26.622494 11346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:26.622761 11346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:26.639175 11346 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40775
I20260929 02:52:26.639195 11412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40775 every 8 connection(s)
I20260929 02:52:26.640560 11346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:26.644182 11413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:26.649953 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11346
I20260929 02:52:26.650141 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:26.650738 8420 external_mini_cluster.cc:1428] Setting key 36d2b6cafa3fee957bf71b7604b4a389
I20260929 02:52:26.654779 11413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e: Bootstrap starting.
I20260929 02:52:26.658310 11413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:26.659379 11413 log.cc:824] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:26.662282 11413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e: No bootstrap required, opened a new log
Sep 29 02:52:26 dist-test-slave-1dkt krb5kdc[11333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:26.670758 11413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40775 } }
I20260929 02:52:26.671202 11413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:26.671324 11413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e547ab8ce05497da5e50c0e8cf8df9e, State: Initialized, Role: FOLLOWER
I20260929 02:52:26.671929 11413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40775 } }
I20260929 02:52:26.672151 11413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:26.672287 11413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:26.672466 11413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:26.672984 11416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:26.652527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.8.57.62:40775)
0929 02:52:26.653274 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:26.653286 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:26.653324 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:52:26.655043 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:26.655066 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:26.655453 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:26.655784 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:26.657580 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:26.658885 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:26.660451 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:26.661127 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:26.661254 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:26.665748 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:26.665756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:26.665771 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:26.665809 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:26.668769 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:26.669542 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:26.669548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:26.669554 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:26.669706 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:26.670059 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:26.670062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:26.670064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:26.670603 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:26.670798 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:26.671333 (+ 535us) server_negotiation.cc:299] Negotiation successful
0929 02:52:26.671781 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":343,"threads_started":1}
I20260929 02:52:26.674707 11413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40775 } }
I20260929 02:52:26.675158 11413 leader_election.cc:302] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e547ab8ce05497da5e50c0e8cf8df9e; no voters:
I20260929 02:52:26.675736 11413 leader_election.cc:288] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:26.675864 11418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:26.676849 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:52:26.676935 11418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [term 1 LEADER]: Becoming Leader. State: Replica: 3e547ab8ce05497da5e50c0e8cf8df9e, State: Running, Role: LEADER
I20260929 02:52:26.677431 11418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40775 } }
I20260929 02:52:26.677775 11413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:26.683543 11420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e547ab8ce05497da5e50c0e8cf8df9e. Latest consensus state: current_term: 1 leader_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40775 } } }
I20260929 02:52:26.683934 11420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:26.683522 11419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e547ab8ce05497da5e50c0e8cf8df9e" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40775 } } }
I20260929 02:52:26.684549 11419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:26.695576 11428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:26.701300 11428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:26.714843 11428 catalog_manager.cc:1347] Generated new cluster ID: 0285cb85660f4831a632657924d65e6f
I20260929 02:52:26.714980 11428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:26.724766 11428 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:26.725951 11428 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:26.731036 11416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:26.719984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.8.57.62:40775)
0929 02:52:26.720121 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:26.720125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:26.720715 (+ 590us) server_negotiation.cc:407] Connection header received
0929 02:52:26.720895 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:26.720900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:26.720983 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:26.721089 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:26.722271 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:26.723099 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:26.727473 (+ 4374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:26.728128 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:26.728249 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:26.729446 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:26.729453 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:26.729458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:26.730212 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:26.730283 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:26.730313 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:26.730403 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:26.730558 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 02:52:26.730689 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 02:52:26.733925 11428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e547ab8ce05497da5e50c0e8cf8df9e: Generated new TSK 0
I20260929 02:52:26.734712 11428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:52:26.742595 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11346
I20260929 02:52:26.743991 11408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:26.829873 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11346
2026-09-29T02:52:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1748 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:26 dist-test-slave-1dkt krb5kdc[11447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:26 dist-test-slave-1dkt krb5kdc[11447](info): set up 2 sockets
Sep 29 02:52:26 dist-test-slave-1dkt krb5kdc[11447](info): commencing operation
krb5kdc: starting...
W20260929 02:52:27.824047 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:27 dist-test-slave-1dkt krb5kdc[11447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:27.996930 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:46237
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:46237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:28.306190 11460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:28.306481 11460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:28.306609 11460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:28.313796 11460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:28.313903 11460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:28.313982 11460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:28.314054 11460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:28.314111 11460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:28.323992 11460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:46237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:46237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:28.325294 11460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:28.326591 11460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:28.337610 11465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:28.337565 11466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:28.337639 11468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:28.338275 11460 server_base.cc:1034] running on GCE node
I20260929 02:52:28.339001 11460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:28.340391 11460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:28.341586 11460 hybrid_clock.cc:648] HybridClock initialized: now 1790650348341535 us; error 36 us; skew 500 ppm
I20260929 02:52:28.341909 11460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:28 dist-test-slave-1dkt krb5kdc[11447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:28.347543 11460 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:28.349116 11460 webserver.cc:460] Webserver started at http://127.8.57.62:33115/ using document root <none> and password file <none>
I20260929 02:52:28.349761 11460 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:28.349874 11460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:28.350158 11460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:28.353148 11460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "62a153b7c56c449b879fa011be13d702"
format_stamp: "Formatted at 2026-09-29 02:52:28 on dist-test-slave-1dkt"
server_key: "ca4236a237a6a395c6f5a77fd365ced2"
server_key_iv: "e6893d8611650d64a68586f55d0c74ae"
server_key_version: "encryptionkey@0"
I20260929 02:52:28.353857 11460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "62a153b7c56c449b879fa011be13d702"
format_stamp: "Formatted at 2026-09-29 02:52:28 on dist-test-slave-1dkt"
server_key: "ca4236a237a6a395c6f5a77fd365ced2"
server_key_iv: "e6893d8611650d64a68586f55d0c74ae"
server_key_version: "encryptionkey@0"
I20260929 02:52:28.358736 11460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:52:28.362000 11475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:28.363278 11460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:28.363423 11460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "62a153b7c56c449b879fa011be13d702"
format_stamp: "Formatted at 2026-09-29 02:52:28 on dist-test-slave-1dkt"
server_key: "ca4236a237a6a395c6f5a77fd365ced2"
server_key_iv: "e6893d8611650d64a68586f55d0c74ae"
server_key_version: "encryptionkey@0"
I20260929 02:52:28.363591 11460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:28.399581 11460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:28.403465 11460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:28.403720 11460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:28.420701 11460 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:46237
I20260929 02:52:28.420719 11526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:46237 every 8 connection(s)
I20260929 02:52:28.422009 11460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:28.422993 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11460
I20260929 02:52:28.423187 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:28.423795 8420 external_mini_cluster.cc:1428] Setting key e0681c881d8c89bfecdf8d55f94fe4f8
I20260929 02:52:28.426115 11527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:52:28 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:28.439181 11527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702: Bootstrap starting.
I20260929 02:52:28.442438 11527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:28.443645 11527 log.cc:824] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:28.446837 11527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702: No bootstrap required, opened a new log
I20260929 02:52:28.447091 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:28.425708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50766 (local address 127.8.57.62:46237)
0929 02:52:28.426485 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:28.426500 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:28.426549 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:52:28.428814 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:28.428846 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:28.429424 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:28.429946 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:28.431378 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:28.433463 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:28.434723 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:28.435748 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:28.435969 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:28.439953 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:28.439962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:28.439977 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:28.440019 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:28.442873 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:28.443662 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:28.443668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:28.443675 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:28.443863 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:28.444325 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:28.444329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:28.444332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:28.445056 (+ 724us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:28.445288 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:28.445717 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 02:52:28.446050 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":318,"threads_started":1}
I20260929 02:52:28.454059 11527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a153b7c56c449b879fa011be13d702" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46237 } }
I20260929 02:52:28.454452 11527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:28.454574 11527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a153b7c56c449b879fa011be13d702, State: Initialized, Role: FOLLOWER
I20260929 02:52:28.455183 11527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a153b7c56c449b879fa011be13d702" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46237 } }
I20260929 02:52:28.455391 11527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:28.455493 11527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:28.455615 11527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:28.457717 11527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a153b7c56c449b879fa011be13d702" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46237 } }
I20260929 02:52:28.458204 11527 leader_election.cc:302] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62a153b7c56c449b879fa011be13d702; no voters:
I20260929 02:52:28.458778 11527 leader_election.cc:288] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:28.458905 11532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:28.459524 11532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [term 1 LEADER]: Becoming Leader. State: Replica: 62a153b7c56c449b879fa011be13d702, State: Running, Role: LEADER
I20260929 02:52:28.460129 11532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a153b7c56c449b879fa011be13d702" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46237 } }
I20260929 02:52:28.460805 11527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:28.462689 11534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62a153b7c56c449b879fa011be13d702. Latest consensus state: current_term: 1 leader_uuid: "62a153b7c56c449b879fa011be13d702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a153b7c56c449b879fa011be13d702" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46237 } } }
I20260929 02:52:28.462998 11534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:28.464596 11537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:28.464486 11533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62a153b7c56c449b879fa011be13d702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a153b7c56c449b879fa011be13d702" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46237 } } }
I20260929 02:52:28.464829 11533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:28.470528 11537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:28.479480 11537 catalog_manager.cc:1347] Generated new cluster ID: 5b1a714310da476193b3fd1f0820eb5b
I20260929 02:52:28.479601 11537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:28.494004 11537 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:28.495271 11537 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:28.505726 11537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62a153b7c56c449b879fa011be13d702: Generated new TSK 0
I20260929 02:52:28.506632 11537 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:28.551530 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46237
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:28.866658 11553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:28.866971 11553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:28.867112 11553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:28.874478 11553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:28.874608 11553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:28.874761 11553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:28.884608 11553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:28.885908 11553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:28.887202 11553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:28.899799 11558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:28.899773 11559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:28.899736 11561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:28.900446 11553 server_base.cc:1034] running on GCE node
I20260929 02:52:28.900983 11553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:28.901816 11553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:28.902999 11553 hybrid_clock.cc:648] HybridClock initialized: now 1790650348902943 us; error 48 us; skew 500 ppm
I20260929 02:52:28.903261 11553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:28 dist-test-slave-1dkt krb5kdc[11447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:28.906149 11553 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:28.907510 11553 webserver.cc:460] Webserver started at http://127.8.57.1:45501/ using document root <none> and password file <none>
I20260929 02:52:28.908228 11553 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:28.908339 11553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:28.908658 11553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:28.911419 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "cc3ded51bc064ca68eede79ce8821460"
format_stamp: "Formatted at 2026-09-29 02:52:28 on dist-test-slave-1dkt"
server_key: "7d5a9027b140ca5062aff885595262b2"
server_key_iv: "0d1d88ac22cb8544813a3628ab349ef3"
server_key_version: "encryptionkey@0"
I20260929 02:52:28.912207 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "cc3ded51bc064ca68eede79ce8821460"
format_stamp: "Formatted at 2026-09-29 02:52:28 on dist-test-slave-1dkt"
server_key: "7d5a9027b140ca5062aff885595262b2"
server_key_iv: "0d1d88ac22cb8544813a3628ab349ef3"
server_key_version: "encryptionkey@0"
I20260929 02:52:28.917021 11553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:52:28.920302 11568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:28.921548 11553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:52:28.921687 11553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "cc3ded51bc064ca68eede79ce8821460"
format_stamp: "Formatted at 2026-09-29 02:52:28 on dist-test-slave-1dkt"
server_key: "7d5a9027b140ca5062aff885595262b2"
server_key_iv: "0d1d88ac22cb8544813a3628ab349ef3"
server_key_version: "encryptionkey@0"
I20260929 02:52:28.921852 11553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:28.946962 11553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:28.950781 11553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:28.951046 11553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:28.952006 11553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:28.953280 11553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:28.953367 11553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:28.953490 11553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:28.953543 11553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:28.982808 11553 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:35283
I20260929 02:52:28.982848 11680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:35283 every 8 connection(s)
I20260929 02:52:28.984102 11553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:28.992914 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11553
I20260929 02:52:28.993099 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:28.993700 8420 external_mini_cluster.cc:1428] Setting key 5770ba0d9b6ae07a4885d2af73784898
Sep 29 02:52:28 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:29.003041 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:28.986778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:56203 (local address 127.8.57.62:46237)
0929 02:52:28.987234 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:28.987238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:28.988022 (+ 784us) server_negotiation.cc:407] Connection header received
0929 02:52:28.989855 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:28.989862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:28.989942 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:28.990036 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:28.991541 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:28.992301 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:28.993891 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:28.994284 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:28.994391 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:28.997904 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:28.997908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:28.997911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:28.997945 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:29.000265 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:29.000934 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:29.000939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:29.000941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:29.001057 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:29.001444 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:29.001448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:29.001451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:29.001712 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:29.001839 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:29.002480 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 02:52:29.002692 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":246,"threads_started":1}
I20260929 02:52:29.004093 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:28.987172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46237 (local address 127.8.57.1:56203)
0929 02:52:28.987850 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:28.987911 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:28.989574 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:28.990293 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:28.990302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:28.990808 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:28.991374 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:28.991391 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:28.992443 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:28.992447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:28.993733 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:28.993745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:28.994495 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:28.994503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:28.994810 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:28.995543 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:28.995582 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:28.997689 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:29.000391 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:29.000400 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:29.000418 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:29.000827 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:29.001162 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:29.001167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:29.001171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:29.001345 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:29.001834 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:29.001842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:29.002152 (+ 310us) client_negotiation.cc:769] Sending connection context
0929 02:52:29.002389 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 02:52:29.002939 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":261,"threads_started":1}
I20260929 02:52:29.005699 11681 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46237
I20260929 02:52:29.006031 11681 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:29.006721 11681 heartbeater.cc:511] Master 127.8.57.62:46237 requested a full tablet report, sending...
I20260929 02:52:29.009150 11492 ts_manager.cc:194] Registered new tserver with Master: cc3ded51bc064ca68eede79ce8821460 (127.8.57.1:35283)
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:29.010874 11492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:56203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:29.032987 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46237
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:29.345286 11687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:29.345559 11687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:29.345680 11687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:29.352825 11687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:29.352934 11687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:29.353085 11687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:29.362792 11687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:29.364090 11687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:29.365373 11687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:29.377632 11692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:29.377599 11693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:29.377564 11695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:29.378506 11687 server_base.cc:1034] running on GCE node
I20260929 02:52:29.379091 11687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:29.380088 11687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:29.381295 11687 hybrid_clock.cc:648] HybridClock initialized: now 1790650349381247 us; error 40 us; skew 500 ppm
I20260929 02:52:29.381601 11687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:29 dist-test-slave-1dkt krb5kdc[11447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:29.384730 11687 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:29.386171 11687 webserver.cc:460] Webserver started at http://127.8.57.2:41247/ using document root <none> and password file <none>
I20260929 02:52:29.386847 11687 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:29.386952 11687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:29.387243 11687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:29.390120 11687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "78ac60da556748089085f5420db87945"
format_stamp: "Formatted at 2026-09-29 02:52:29 on dist-test-slave-1dkt"
server_key: "5c94c1d883d0fc8d0e0b085227dfcacd"
server_key_iv: "626c42d1d5b5268554b2f4aa893b14ac"
server_key_version: "encryptionkey@0"
I20260929 02:52:29.390887 11687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "78ac60da556748089085f5420db87945"
format_stamp: "Formatted at 2026-09-29 02:52:29 on dist-test-slave-1dkt"
server_key: "5c94c1d883d0fc8d0e0b085227dfcacd"
server_key_iv: "626c42d1d5b5268554b2f4aa893b14ac"
server_key_version: "encryptionkey@0"
I20260929 02:52:29.395946 11687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:52:29.399317 11702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:29.400678 11687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:52:29.400826 11687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "78ac60da556748089085f5420db87945"
format_stamp: "Formatted at 2026-09-29 02:52:29 on dist-test-slave-1dkt"
server_key: "5c94c1d883d0fc8d0e0b085227dfcacd"
server_key_iv: "626c42d1d5b5268554b2f4aa893b14ac"
server_key_version: "encryptionkey@0"
I20260929 02:52:29.401005 11687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:29.424423 11687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:29.429255 11687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:29.429591 11687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:29.430685 11687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:29.432292 11687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:29.432391 11687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:29.432497 11687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:29.432549 11687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:29.461297 11687 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:34529
I20260929 02:52:29.461349 11814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:34529 every 8 connection(s)
I20260929 02:52:29.462582 11687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:29.469614 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11687
I20260929 02:52:29.469755 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:29.470196 8420 external_mini_cluster.cc:1428] Setting key 76beebf2a9fad6a7242122780df5e0e7
Sep 29 02:52:29 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:29.481840 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:29.465102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:51345 (local address 127.8.57.62:46237)
0929 02:52:29.465278 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:29.465282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:29.466582 (+ 1300us) server_negotiation.cc:407] Connection header received
0929 02:52:29.468505 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:29.468511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:29.468595 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:29.468682 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:29.470257 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.471078 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.472680 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.473096 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.473203 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:29.476450 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:29.476455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:29.476458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:29.476489 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:29.478897 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:29.479697 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:29.479701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:29.479703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:29.479827 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:29.480288 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:29.480291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:29.480293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:29.480557 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:29.480671 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:29.481430 (+ 759us) server_negotiation.cc:299] Negotiation successful
0929 02:52:29.481613 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:52:29.483253 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:29.465644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46237 (local address 127.8.57.2:51345)
0929 02:52:29.466424 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:29.466471 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:29.468233 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:29.468926 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:29.468934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:29.469457 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:29.470084 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:29.470104 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.471222 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.471225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:29.472550 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:29.472563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.473295 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.473302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:29.473519 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:29.474131 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:29.474152 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:29.476249 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:29.479036 (+ 2787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:29.479045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:29.479061 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:29.479592 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:29.479956 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:29.479961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:29.479964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:29.480191 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:29.480661 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:29.480668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:29.481082 (+ 414us) client_negotiation.cc:769] Sending connection context
0929 02:52:29.481364 (+ 282us) client_negotiation.cc:240] Negotiation successful
0929 02:52:29.481888 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":312,"threads_started":1}
I20260929 02:52:29.484656 11815 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46237
I20260929 02:52:29.484946 11815 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:29.485641 11815 heartbeater.cc:511] Master 127.8.57.62:46237 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:29.487066 11492 ts_manager.cc:194] Registered new tserver with Master: 78ac60da556748089085f5420db87945 (127.8.57.2:34529)
I20260929 02:52:29.487928 11492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:51345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:29.510278 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46237
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:29.821630 11820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:29.821914 11820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:29.822060 11820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:29.829445 11820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:29.829563 11820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:29.829710 11820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:29.839547 11820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:29.840855 11820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:29.842144 11820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:29.854764 11828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:29.854805 11825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:29.854827 11826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:29.855647 11820 server_base.cc:1034] running on GCE node
I20260929 02:52:29.856259 11820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:29.857173 11820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:29.858373 11820 hybrid_clock.cc:648] HybridClock initialized: now 1790650349858316 us; error 49 us; skew 500 ppm
I20260929 02:52:29.858690 11820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:29 dist-test-slave-1dkt krb5kdc[11447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:29.861821 11820 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:29.863240 11820 webserver.cc:460] Webserver started at http://127.8.57.3:45203/ using document root <none> and password file <none>
I20260929 02:52:29.863957 11820 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:29.864075 11820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:29.864360 11820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:29.867511 11820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "cdc882dee3bc499eb7d4f46e5d6decdd"
format_stamp: "Formatted at 2026-09-29 02:52:29 on dist-test-slave-1dkt"
server_key: "a45fc4949184b65bf7c879f02b8e9191"
server_key_iv: "fd0bb46d0a92ceeaf808a50e87f37c73"
server_key_version: "encryptionkey@0"
I20260929 02:52:29.868285 11820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "cdc882dee3bc499eb7d4f46e5d6decdd"
format_stamp: "Formatted at 2026-09-29 02:52:29 on dist-test-slave-1dkt"
server_key: "a45fc4949184b65bf7c879f02b8e9191"
server_key_iv: "fd0bb46d0a92ceeaf808a50e87f37c73"
server_key_version: "encryptionkey@0"
I20260929 02:52:29.873337 11820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 02:52:29.876739 11835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:29.878134 11820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:29.878285 11820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "cdc882dee3bc499eb7d4f46e5d6decdd"
format_stamp: "Formatted at 2026-09-29 02:52:29 on dist-test-slave-1dkt"
server_key: "a45fc4949184b65bf7c879f02b8e9191"
server_key_iv: "fd0bb46d0a92ceeaf808a50e87f37c73"
server_key_version: "encryptionkey@0"
I20260929 02:52:29.878495 11820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:29.892347 11820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:29.896622 11820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:29.896895 11820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:29.897818 11820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:29.899168 11820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:29.899250 11820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:29.899365 11820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:29.899430 11820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:29.928390 11820 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:42869
I20260929 02:52:29.928426 11947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:42869 every 8 connection(s)
I20260929 02:52:29.929608 11820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:29.937299 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11820
I20260929 02:52:29.937507 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:29.938042 8420 external_mini_cluster.cc:1428] Setting key 8e75eebebbae9c71dde253da01a4bbbb
Sep 29 02:52:29 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:29.948246 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:29.932028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:50085 (local address 127.8.57.62:46237)
0929 02:52:29.932197 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:29.932201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:29.933342 (+ 1141us) server_negotiation.cc:407] Connection header received
0929 02:52:29.935161 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:29.935166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:29.935244 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:29.935339 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:29.936832 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.937884 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.939607 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.940106 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.940225 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:29.943379 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:29.943383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:29.943387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:29.943418 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:29.945756 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:29.946430 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:29.946434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:29.946436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:29.946548 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:29.946867 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:29.946870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:29.946872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:29.947125 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:29.947228 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:29.947791 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 02:52:29.948010 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:52:29.949239 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:29.932492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46237 (local address 127.8.57.3:50085)
0929 02:52:29.933170 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:29.933236 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:29.934877 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:29.935583 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:29.935591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:29.936124 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:29.936677 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:29.936695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.938043 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.938047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:29.939478 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:29.939490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.940303 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.940308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:29.940519 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:29.941120 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:29.941141 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:29.943196 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:29.945901 (+ 2705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:29.945907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:29.945919 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:29.946345 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:29.946624 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:29.946627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:29.946630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:29.946795 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:29.947211 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:29.947216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:29.947514 (+ 298us) client_negotiation.cc:769] Sending connection context
0929 02:52:29.947727 (+ 213us) client_negotiation.cc:240] Negotiation successful
0929 02:52:29.948160 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":271,"threads_started":1}
I20260929 02:52:29.950515 11948 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46237
I20260929 02:52:29.950796 11948 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:29.951504 11948 heartbeater.cc:511] Master 127.8.57.62:46237 requested a full tablet report, sending...
I20260929 02:52:29.953020 11491 ts_manager.cc:194] Registered new tserver with Master: cdc882dee3bc499eb7d4f46e5d6decdd (127.8.57.3:42869)
I20260929 02:52:29.953843 11491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:50085
I20260929 02:52:29.965173 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:52:29 dist-test-slave-1dkt krb5kdc[11447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 02:52:29 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:30.008932 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:29.995814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.8.57.62:46237)
0929 02:52:29.996072 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:29.996078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:29.996363 (+ 285us) server_negotiation.cc:407] Connection header received
0929 02:52:29.996517 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:29.996521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:29.996584 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:29.996667 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:29.997892 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:29.998628 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:29.999762 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.000252 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.000343 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:30.004230 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:30.004235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:30.004238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:30.004275 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:30.006745 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.007339 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.007342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.007344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.007456 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.007742 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.007744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.007746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.008092 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:30.008268 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:30.008498 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:52:30.008659 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260929 02:52:30.009473 11491 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50774
I20260929 02:52:30.010952 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11553
I20260929 02:52:30.013763 11681 heartbeater.cc:503] Master 127.8.57.62:46237 was elected leader, sending a full tablet report...
Sep 29 02:52:30 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:30.025909 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:30.012051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.8.57.1:35283)
0929 02:52:30.012589 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:30.012597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:30.012621 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:52:30.012676 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:30.012682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:30.012880 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:30.013011 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:30.014086 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.015105 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.016330 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.016881 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.016993 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:30.020662 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:30.020673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:30.020687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:30.020721 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:30.023232 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.023841 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.023848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.023851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.024025 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.024367 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.024372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.024376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.024726 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:30.024997 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:30.025295 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 02:52:30.025556 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":254,"threads_started":1}
W20260929 02:52:30.027186 11676 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36300
W20260929 02:52:30.028294 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11553: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:30.028498 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11553
I20260929 02:52:30.040337 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11687
Sep 29 02:52:30 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:30.055140 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:30.041160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.8.57.2:34529)
0929 02:52:30.041694 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:30.041701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:30.041725 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:52:30.041783 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:30.041789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:30.041998 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:30.042127 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:30.043201 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.044243 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.045425 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.045971 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.046071 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:30.050027 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:30.050038 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:30.050052 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:30.050094 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:30.052822 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.053449 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.053456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.053458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.053586 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.053883 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.053886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.053888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.054231 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:30.054453 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:30.054645 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 02:52:30.054841 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":286,"threads_started":1}
W20260929 02:52:30.056422 11810 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47106
W20260929 02:52:30.057487 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:30.057617 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11687
I20260929 02:52:30.069171 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11820
Sep 29 02:52:30 dist-test-slave-1dkt krb5kdc[11447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650349, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:30.084077 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:30.070034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.8.57.3:42869)
0929 02:52:30.070629 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:30.070636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:30.070663 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:52:30.070745 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:30.070752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:30.070956 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:30.071096 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:30.072449 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.073470 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.074655 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:30.075191 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:30.075299 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:30.079250 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:30.079260 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:30.079275 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:30.079308 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:30.081800 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.082375 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.082381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.082384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.082518 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:30.082787 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:30.082790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:30.082792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:30.083142 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:30.083371 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:30.083579 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 02:52:30.083761 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":307,"threads_started":1}
W20260929 02:52:30.085245 11943 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53456
W20260929 02:52:30.086268 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11820: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:52:30.086413 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11820
I20260929 02:52:30.097745 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11460
I20260929 02:52:30.098942 11522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:30.202448 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11460
2026-09-29T02:52:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3377 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:30 dist-test-slave-1dkt krb5kdc[11965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:30 dist-test-slave-1dkt krb5kdc[11965](info): set up 2 sockets
Sep 29 02:52:30 dist-test-slave-1dkt krb5kdc[11965](info): commencing operation
krb5kdc: starting...
W20260929 02:52:31.208117 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:31 dist-test-slave-1dkt krb5kdc[11965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:31.385483 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40101
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:31.695041 11978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:31.695328 11978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:31.695446 11978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:31.702781 11978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:31.702894 11978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:31.702965 11978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:31.703045 11978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:31.703105 11978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:31.713133 11978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:31.714428 11978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:31.715728 11978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:31.727138 11983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:31.727106 11984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:31.727232 11986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:31.728212 11978 server_base.cc:1034] running on GCE node
I20260929 02:52:31.728986 11978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:31.730402 11978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:31.731606 11978 hybrid_clock.cc:648] HybridClock initialized: now 1790650351731550 us; error 49 us; skew 500 ppm
I20260929 02:52:31.731989 11978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:31 dist-test-slave-1dkt krb5kdc[11965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650351, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:31.734917 11978 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:31.736341 11978 webserver.cc:460] Webserver started at http://127.8.57.62:34705/ using document root <none> and password file <none>
I20260929 02:52:31.736985 11978 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:31.737083 11978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:31.737336 11978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:31.740309 11978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "23600150bd2f470c9ced610c76e00ff3"
format_stamp: "Formatted at 2026-09-29 02:52:31 on dist-test-slave-1dkt"
server_key: "9bfdec2b89a0b80e7c9239ab175aaebd"
server_key_iv: "777bf3aba1bbe066e5afa049cefc445c"
server_key_version: "encryptionkey@0"
I20260929 02:52:31.741053 11978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "23600150bd2f470c9ced610c76e00ff3"
format_stamp: "Formatted at 2026-09-29 02:52:31 on dist-test-slave-1dkt"
server_key: "9bfdec2b89a0b80e7c9239ab175aaebd"
server_key_iv: "777bf3aba1bbe066e5afa049cefc445c"
server_key_version: "encryptionkey@0"
I20260929 02:52:31.746105 11978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:31.749552 11993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:31.750897 11978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:52:31.751049 11978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "23600150bd2f470c9ced610c76e00ff3"
format_stamp: "Formatted at 2026-09-29 02:52:31 on dist-test-slave-1dkt"
server_key: "9bfdec2b89a0b80e7c9239ab175aaebd"
server_key_iv: "777bf3aba1bbe066e5afa049cefc445c"
server_key_version: "encryptionkey@0"
I20260929 02:52:31.751227 11978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:31.778931 11978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:31.783000 11978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:31.783279 11978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:31.800222 11978 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40101
I20260929 02:52:31.800246 12044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40101 every 8 connection(s)
I20260929 02:52:31.801599 11978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:31.802790 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 11978
I20260929 02:52:31.803043 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:31.803803 8420 external_mini_cluster.cc:1428] Setting key b1d7c601a38a922456b813813d708497
I20260929 02:52:31.806214 12045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:52:31 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:31.818732 12045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3: Bootstrap starting.
I20260929 02:52:31.822782 12045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:31.823956 12045 log.cc:824] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:31.827250 12045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3: No bootstrap required, opened a new log
I20260929 02:52:31.827353 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:31.805821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.8.57.62:40101)
0929 02:52:31.806603 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:31.806614 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:31.806654 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:52:31.808276 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:31.808297 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:31.808697 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:31.809032 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:31.810763 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:31.812148 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:31.813744 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:31.814394 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:31.814509 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:31.818950 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:31.818961 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:31.818980 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:31.819025 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:31.822843 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:31.823584 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:31.823597 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:31.823606 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:31.823809 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:31.824180 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:31.824185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:31.824188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:31.824877 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:31.825106 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:31.825604 (+ 498us) server_negotiation.cc:299] Negotiation successful
0929 02:52:31.826081 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":341,"threads_started":1}
I20260929 02:52:31.835041 12045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23600150bd2f470c9ced610c76e00ff3" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40101 } }
I20260929 02:52:31.835397 12045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:31.835476 12045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23600150bd2f470c9ced610c76e00ff3, State: Initialized, Role: FOLLOWER
I20260929 02:52:31.836069 12045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23600150bd2f470c9ced610c76e00ff3" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40101 } }
I20260929 02:52:31.836267 12045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:31.836359 12045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:31.836486 12045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:31.838608 12045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23600150bd2f470c9ced610c76e00ff3" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40101 } }
I20260929 02:52:31.839182 12045 leader_election.cc:302] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23600150bd2f470c9ced610c76e00ff3; no voters:
I20260929 02:52:31.839764 12045 leader_election.cc:288] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:31.839854 12050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:31.840529 12050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [term 1 LEADER]: Becoming Leader. State: Replica: 23600150bd2f470c9ced610c76e00ff3, State: Running, Role: LEADER
I20260929 02:52:31.841089 12050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23600150bd2f470c9ced610c76e00ff3" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40101 } }
I20260929 02:52:31.841751 12045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:31.843750 12051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23600150bd2f470c9ced610c76e00ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23600150bd2f470c9ced610c76e00ff3" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40101 } } }
I20260929 02:52:31.844131 12051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:31.844108 12052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23600150bd2f470c9ced610c76e00ff3. Latest consensus state: current_term: 1 leader_uuid: "23600150bd2f470c9ced610c76e00ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23600150bd2f470c9ced610c76e00ff3" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40101 } } }
I20260929 02:52:31.844412 12052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:31.847467 12058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:31.852557 12058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:31.862039 12058 catalog_manager.cc:1347] Generated new cluster ID: 25f63e0d911d40cab9c19f66d46d7b08
I20260929 02:52:31.862160 12058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:31.872280 12058 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:31.873647 12058 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:31.879114 12058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23600150bd2f470c9ced610c76e00ff3: Generated new TSK 0
I20260929 02:52:31.879961 12058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:31.931974 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40101
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:32.272666 12071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:32.272943 12071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:32.273033 12071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:32.280287 12071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:32.280395 12071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:32.280512 12071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:32.290488 12071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:32.291716 12071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:32.293069 12071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:32.305753 12076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:32.305675 12077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:32.305655 12079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:32.307000 12071 server_base.cc:1034] running on GCE node
I20260929 02:52:32.307580 12071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:32.308583 12071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:32.309778 12071 hybrid_clock.cc:648] HybridClock initialized: now 1790650352309725 us; error 50 us; skew 500 ppm
I20260929 02:52:32.310101 12071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:32 dist-test-slave-1dkt krb5kdc[11965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:32.313256 12071 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:32.314795 12071 webserver.cc:460] Webserver started at http://127.8.57.1:41881/ using document root <none> and password file <none>
I20260929 02:52:32.315464 12071 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:32.315567 12071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:32.315817 12071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:32.318758 12071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "8950a0cdf6354650bf381f734aef2ff4"
format_stamp: "Formatted at 2026-09-29 02:52:32 on dist-test-slave-1dkt"
server_key: "981d04a09427cce0feab24782a05d5b1"
server_key_iv: "286d8860e24632a0572a17e20f6ee1af"
server_key_version: "encryptionkey@0"
I20260929 02:52:32.319530 12071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "8950a0cdf6354650bf381f734aef2ff4"
format_stamp: "Formatted at 2026-09-29 02:52:32 on dist-test-slave-1dkt"
server_key: "981d04a09427cce0feab24782a05d5b1"
server_key_iv: "286d8860e24632a0572a17e20f6ee1af"
server_key_version: "encryptionkey@0"
I20260929 02:52:32.324959 12071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 02:52:32.328804 12086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:32.330121 12071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:52:32.330250 12071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "8950a0cdf6354650bf381f734aef2ff4"
format_stamp: "Formatted at 2026-09-29 02:52:32 on dist-test-slave-1dkt"
server_key: "981d04a09427cce0feab24782a05d5b1"
server_key_iv: "286d8860e24632a0572a17e20f6ee1af"
server_key_version: "encryptionkey@0"
I20260929 02:52:32.330425 12071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:32.344280 12071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:32.348317 12071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:32.348572 12071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:32.349426 12071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:32.350669 12071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:32.350739 12071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:32.350831 12071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:32.350874 12071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:32.379786 12071 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:39445
I20260929 02:52:32.379925 12198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:39445 every 8 connection(s)
I20260929 02:52:32.381095 12071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:32.382208 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12071
I20260929 02:52:32.382407 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:32.383165 8420 external_mini_cluster.cc:1428] Setting key b2372e8abe0de6cad4810e52002fff9b
Sep 29 02:52:32 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:32.401363 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:32.383810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:37559 (local address 127.8.57.62:40101)
0929 02:52:32.385792 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:32.385798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:32.385819 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:52:32.387134 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:32.387140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:32.387217 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:32.387304 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:32.388809 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.389577 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.391064 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.391662 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.391777 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:32.395240 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:32.395246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:32.395250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:32.395286 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:32.398526 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:32.399282 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:32.399288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:32.399292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:32.399427 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:32.399807 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:32.399811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:32.399814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:32.400178 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:32.400294 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:32.400944 (+ 650us) server_negotiation.cc:299] Negotiation successful
0929 02:52:32.401112 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1872,"thread_start_us":1769,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:52:32.402535 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:32.384357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40101 (local address 127.8.57.1:37559)
0929 02:52:32.385107 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:32.385151 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:32.386877 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:32.387549 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:32.387558 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:32.388104 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:32.388651 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:32.388669 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.389733 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.389737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:32.390932 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:32.390944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.391866 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.391893 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:32.392106 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:32.392739 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:32.392760 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:32.395010 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:32.398704 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:32.398712 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:32.398726 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:32.399150 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:32.399524 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:32.399527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:32.399529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:32.399704 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:32.400288 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:32.400294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:32.400622 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 02:52:32.400837 (+ 215us) client_negotiation.cc:240] Negotiation successful
0929 02:52:32.401298 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":301,"threads_started":1}
I20260929 02:52:32.405346 12199 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40101
I20260929 02:52:32.405686 12199 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:32.406545 12199 heartbeater.cc:511] Master 127.8.57.62:40101 requested a full tablet report, sending...
I20260929 02:52:32.408867 12010 ts_manager.cc:194] Registered new tserver with Master: 8950a0cdf6354650bf381f734aef2ff4 (127.8.57.1:39445)
I20260929 02:52:32.410743 12010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:37559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:32.424389 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40101
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:32.729867 12205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:32.730134 12205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:32.730223 12205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:32.737305 12205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:32.737409 12205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:32.737517 12205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:32.747094 12205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:32.748337 12205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:32.749668 12205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:32.765455 12210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:32.765475 12211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:32.765597 12213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:32.766742 12205 server_base.cc:1034] running on GCE node
I20260929 02:52:32.767289 12205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:32.768177 12205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:32.769364 12205 hybrid_clock.cc:648] HybridClock initialized: now 1790650352769320 us; error 39 us; skew 500 ppm
I20260929 02:52:32.769642 12205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:32 dist-test-slave-1dkt krb5kdc[11965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:32.772462 12205 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:32.773774 12205 webserver.cc:460] Webserver started at http://127.8.57.2:42829/ using document root <none> and password file <none>
I20260929 02:52:32.774399 12205 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:32.774503 12205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:32.774771 12205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:32.777558 12205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "994e459065884bb78ee673232919bddd"
format_stamp: "Formatted at 2026-09-29 02:52:32 on dist-test-slave-1dkt"
server_key: "0273992094e6033f534495bbb11b49b3"
server_key_iv: "4e59a11c0f31c9a2ab34594174f663ab"
server_key_version: "encryptionkey@0"
I20260929 02:52:32.778264 12205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "994e459065884bb78ee673232919bddd"
format_stamp: "Formatted at 2026-09-29 02:52:32 on dist-test-slave-1dkt"
server_key: "0273992094e6033f534495bbb11b49b3"
server_key_iv: "4e59a11c0f31c9a2ab34594174f663ab"
server_key_version: "encryptionkey@0"
I20260929 02:52:32.782920 12205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 02:52:32.786067 12220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:32.787266 12205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:52:32.787411 12205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "994e459065884bb78ee673232919bddd"
format_stamp: "Formatted at 2026-09-29 02:52:32 on dist-test-slave-1dkt"
server_key: "0273992094e6033f534495bbb11b49b3"
server_key_iv: "4e59a11c0f31c9a2ab34594174f663ab"
server_key_version: "encryptionkey@0"
I20260929 02:52:32.787627 12205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:32.810148 12205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:32.814105 12205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:32.814350 12205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:32.815207 12205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:32.816442 12205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:32.816516 12205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:32.816630 12205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:32.816681 12205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:32.845968 12205 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:44803
I20260929 02:52:32.846016 12332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:44803 every 8 connection(s)
I20260929 02:52:32.847188 12205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:32.850594 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12205
I20260929 02:52:32.850780 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:32.851285 8420 external_mini_cluster.cc:1428] Setting key 2859b30abecc2915796ebf919b316399
Sep 29 02:52:32 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650352, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:32.866432 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:32.849649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:49381 (local address 127.8.57.62:40101)
0929 02:52:32.849811 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:32.849816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:32.850973 (+ 1157us) server_negotiation.cc:407] Connection header received
0929 02:52:32.852988 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:32.852994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:32.853080 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:32.853164 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:32.854589 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.855325 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.856839 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.857264 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.857376 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:32.861143 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:32.861148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:32.861152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:32.861183 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:32.863464 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:32.864318 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:32.864322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:32.864324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:32.864443 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:32.864878 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:32.864881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:32.864883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:32.865131 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:32.865232 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:32.866055 (+ 823us) server_negotiation.cc:299] Negotiation successful
0929 02:52:32.866208 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:32.867858 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:32.850094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40101 (local address 127.8.57.2:49381)
0929 02:52:32.850825 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:32.850870 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:32.852720 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:32.853389 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:32.853397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:32.853891 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:32.854439 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:32.854457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.855454 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.855458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:32.856712 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:32.856724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:32.857460 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:32.857467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:32.857685 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:32.858332 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:32.858373 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:32.860937 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:32.863600 (+ 2663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:32.863609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:32.863626 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:32.864203 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:32.864548 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:32.864553 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:32.864557 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:32.864785 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:32.865242 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:32.865249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:32.865668 (+ 419us) client_negotiation.cc:769] Sending connection context
0929 02:52:32.865967 (+ 299us) client_negotiation.cc:240] Negotiation successful
0929 02:52:32.866468 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":289,"threads_started":1}
I20260929 02:52:32.869309 12333 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40101
I20260929 02:52:32.869630 12333 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
I20260929 02:52:32.870466 12333 heartbeater.cc:511] Master 127.8.57.62:40101 requested a full tablet report, sending...
I20260929 02:52:32.871973 12010 ts_manager.cc:194] Registered new tserver with Master: 994e459065884bb78ee673232919bddd (127.8.57.2:44803)
I20260929 02:52:32.872750 12010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:49381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:32.891029 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40101
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:33.197695 12338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:33.198032 12338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:33.198165 12338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:33.205323 12338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:33.205435 12338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:33.205576 12338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:33.215121 12338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:33.216383 12338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:33.217636 12338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:33.229779 12343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:33.229859 12344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:33.229839 12346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:33.230691 12338 server_base.cc:1034] running on GCE node
I20260929 02:52:33.231257 12338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:33.232133 12338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:33.233330 12338 hybrid_clock.cc:648] HybridClock initialized: now 1790650353233290 us; error 42 us; skew 500 ppm
I20260929 02:52:33.233632 12338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:33.236678 12338 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:33.238096 12338 webserver.cc:460] Webserver started at http://127.8.57.3:43541/ using document root <none> and password file <none>
I20260929 02:52:33.238727 12338 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:33.238835 12338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:33.239120 12338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:33.241986 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "fa38cca7daa242e48d4f53e784fc4fe4"
format_stamp: "Formatted at 2026-09-29 02:52:33 on dist-test-slave-1dkt"
server_key: "10fb4a32759afc2ec7309a8e90124fa8"
server_key_iv: "814fc6657fc5078f80485ca8407a7c70"
server_key_version: "encryptionkey@0"
I20260929 02:52:33.242724 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "fa38cca7daa242e48d4f53e784fc4fe4"
format_stamp: "Formatted at 2026-09-29 02:52:33 on dist-test-slave-1dkt"
server_key: "10fb4a32759afc2ec7309a8e90124fa8"
server_key_iv: "814fc6657fc5078f80485ca8407a7c70"
server_key_version: "encryptionkey@0"
I20260929 02:52:33.247568 12338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:52:33.250866 12353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:33.252125 12338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:52:33.252259 12338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "fa38cca7daa242e48d4f53e784fc4fe4"
format_stamp: "Formatted at 2026-09-29 02:52:33 on dist-test-slave-1dkt"
server_key: "10fb4a32759afc2ec7309a8e90124fa8"
server_key_iv: "814fc6657fc5078f80485ca8407a7c70"
server_key_version: "encryptionkey@0"
I20260929 02:52:33.252426 12338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:33.291225 12338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:33.295181 12338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:33.295459 12338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:33.296442 12338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:33.297789 12338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:33.297866 12338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:33.297991 12338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:33.298043 12338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:33.327232 12338 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:35565
I20260929 02:52:33.327293 12465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:35565 every 8 connection(s)
I20260929 02:52:33.328522 12338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:33.337076 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12338
I20260929 02:52:33.337253 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:33.337646 8420 external_mini_cluster.cc:1428] Setting key 3ad160185fb0d604ed1ab0a4ba386582
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:33.346710 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.330743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:45503 (local address 127.8.57.62:40101)
0929 02:52:33.330899 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:33.330903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:33.332057 (+ 1154us) server_negotiation.cc:407] Connection header received
0929 02:52:33.333882 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:33.333886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:33.333958 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:33.334032 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:33.335360 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.336123 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.337770 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.338222 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.338344 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.341715 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:33.341720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:33.341723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:33.341753 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:33.344315 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.344963 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.344966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.344987 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.345098 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.345408 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.345411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.345413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.345655 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:33.345764 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:33.346334 (+ 570us) server_negotiation.cc:299] Negotiation successful
0929 02:52:33.346482 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:52:33.347594 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.331224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40101 (local address 127.8.57.3:45503)
0929 02:52:33.331865 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:33.331948 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:33.333639 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:33.334228 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:33.334235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:33.334708 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:33.335227 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:33.335242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.336233 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.336236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:33.337627 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:33.337639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.338473 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.338480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:33.338721 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.339367 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:33.339394 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:33.341542 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:33.344443 (+ 2901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:33.344449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:33.344460 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:33.344865 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:33.345174 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:33.345177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:33.345179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:33.345335 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:33.345746 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:33.345751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:33.346041 (+ 290us) client_negotiation.cc:769] Sending connection context
0929 02:52:33.346254 (+ 213us) client_negotiation.cc:240] Negotiation successful
0929 02:52:33.346606 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":255,"threads_started":1}
I20260929 02:52:33.348796 12466 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40101
I20260929 02:52:33.349059 12466 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:33.349707 12466 heartbeater.cc:511] Master 127.8.57.62:40101 requested a full tablet report, sending...
I20260929 02:52:33.350963 12010 ts_manager.cc:194] Registered new tserver with Master: fa38cca7daa242e48d4f53e784fc4fe4 (127.8.57.3:35565)
I20260929 02:52:33.351800 12010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:45503
I20260929 02:52:33.352581 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:33.400295 12473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.382229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.8.57.1:39445)
0929 02:52:33.382915 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:33.382923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:33.382948 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:52:33.382998 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:33.383003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:33.383195 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:33.383327 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:33.384678 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.385688 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.386876 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.387395 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.387511 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.394987 (+ 7476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:33.395000 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:33.395013 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:33.395048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:33.397674 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.398317 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.398324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.398326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.398455 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.398795 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.398799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.398801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.399124 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:33.399399 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:33.399682 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 02:52:33.399958 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":386,"threads_started":1}
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:33.413695 12199 heartbeater.cc:503] Master 127.8.57.62:40101 was elected leader, sending a full tablet report...
I20260929 02:52:33.416391 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.404121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.8.57.62:40101)
0929 02:52:33.404311 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:33.404316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:33.404497 (+ 181us) server_negotiation.cc:407] Connection header received
0929 02:52:33.404652 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:33.404656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:33.404717 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:33.404794 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:33.405909 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.406611 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.407812 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.408254 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.408360 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.411781 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:33.411786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:33.411789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:33.411820 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:33.414068 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.414632 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.414638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.414642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.414771 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.415076 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.415081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.415085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.415373 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:33.415597 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:33.415929 (+ 332us) server_negotiation.cc:299] Negotiation successful
0929 02:52:33.416127 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 02:52:33.416834 12009 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51652
I20260929 02:52:33.417663 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12071
I20260929 02:52:33.428700 12473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.418422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37904 (local address 127.8.57.1:39445)
0929 02:52:33.418567 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:33.418571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:33.418769 (+ 198us) server_negotiation.cc:407] Connection header received
0929 02:52:33.418880 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:33.418884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:33.418940 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:33.419033 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:33.420054 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.420688 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.421769 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.422196 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.422286 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.424216 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:33.424220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:33.424224 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:33.424253 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:33.426422 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.426978 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.426984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.426987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.427112 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.427416 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.427421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.427425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.427709 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:33.427959 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:33.428206 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:52:33.428405 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:52:33.429234 12194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:33.509816 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12071
I20260929 02:52:33.521705 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12205
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:33.536392 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.522476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.8.57.2:44803)
0929 02:52:33.522981 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:33.522988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:33.523013 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:52:33.523069 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:33.523076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:33.523279 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:33.523412 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:33.524626 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.525623 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.526870 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.527403 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.527515 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.531364 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:33.531375 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:33.531389 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:33.531428 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:33.533854 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.534449 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.534456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.534458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.534577 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.534901 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.534905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.534907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.535231 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:33.535514 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:33.535796 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:52:33.536073 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":243,"threads_started":1}
I20260929 02:52:33.537708 12328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:33.621495 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12205
I20260929 02:52:33.633123 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12338
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[11965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:33.648563 12480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:33.634022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.8.57.3:35565)
0929 02:52:33.634602 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:33.634608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:33.634636 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:52:33.634695 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:33.634700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:33.634902 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:33.635038 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:33.636300 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.637305 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.638505 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:33.639044 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:33.639150 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:33.643315 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:33.643325 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:33.643339 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:33.643372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:33.645850 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.646468 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.646481 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.646484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.646601 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:33.646933 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:33.646937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:33.646939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:33.647276 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:33.647570 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:33.647866 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 02:52:33.648183 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":277,"threads_started":1}
I20260929 02:52:33.649811 12461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:33.730270 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12338
I20260929 02:52:33.741968 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 11978
I20260929 02:52:33.743099 12040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:33.843111 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 11978
2026-09-29T02:52:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3640 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[12485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[12485](info): set up 2 sockets
Sep 29 02:52:33 dist-test-slave-1dkt krb5kdc[12485](info): commencing operation
krb5kdc: starting...
W20260929 02:52:34.832442 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:34 dist-test-slave-1dkt krb5kdc[12485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:35.009393 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:39893
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:39893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:35.310845 12498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:35.311134 12498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:35.311225 12498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:35.318559 12498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:35.318668 12498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:35.318718 12498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:35.318773 12498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:35.318818 12498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:35.328801 12498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:39893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:35.330009 12498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:35.331257 12498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:35.341869 12504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:35.341863 12503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:35.342001 12506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:35.342643 12498 server_base.cc:1034] running on GCE node
I20260929 02:52:35.343380 12498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:35.344736 12498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:35.345935 12498 hybrid_clock.cc:648] HybridClock initialized: now 1790650355345890 us; error 43 us; skew 500 ppm
I20260929 02:52:35.346257 12498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:35 dist-test-slave-1dkt krb5kdc[12485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:35.349061 12498 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:35.350090 12498 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:35.350185 12498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:35.350476 12498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:35.353281 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "d910d2239fec4e16a5e24a921ef8bf6c"
format_stamp: "Formatted at 2026-09-29 02:52:35 on dist-test-slave-1dkt"
server_key: "9fa97758bfbd9e3e40f1216b0e644b14"
server_key_iv: "94c40fd237e2eed7378acd03f240bc93"
server_key_version: "encryptionkey@0"
I20260929 02:52:35.354014 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "d910d2239fec4e16a5e24a921ef8bf6c"
format_stamp: "Formatted at 2026-09-29 02:52:35 on dist-test-slave-1dkt"
server_key: "9fa97758bfbd9e3e40f1216b0e644b14"
server_key_iv: "94c40fd237e2eed7378acd03f240bc93"
server_key_version: "encryptionkey@0"
I20260929 02:52:35.358566 12498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:52:35.364069 12512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:35.365857 12498 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260929 02:52:35.366030 12498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "d910d2239fec4e16a5e24a921ef8bf6c"
format_stamp: "Formatted at 2026-09-29 02:52:35 on dist-test-slave-1dkt"
server_key: "9fa97758bfbd9e3e40f1216b0e644b14"
server_key_iv: "94c40fd237e2eed7378acd03f240bc93"
server_key_version: "encryptionkey@0"
I20260929 02:52:35.366252 12498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:35.400810 12498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:35.404588 12498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:35.404827 12498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:35.420889 12498 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:39893
I20260929 02:52:35.420914 12563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:39893 every 8 connection(s)
I20260929 02:52:35.421715 12498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:35.424072 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12498
I20260929 02:52:35.424265 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:35.424852 8420 external_mini_cluster.cc:1428] Setting key b5835d729597b4146adb0b41244e613e
I20260929 02:52:35.425315 12564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:35.438997 12564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c: Bootstrap starting.
Sep 29 02:52:35 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:35.442740 12564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:35.444041 12564 log.cc:824] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:35.446651 12564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c: No bootstrap required, opened a new log
I20260929 02:52:35.447829 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:35.426761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.8.57.62:39893)
0929 02:52:35.427539 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:35.427643 (+ 104us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:35.427701 (+ 58us) server_negotiation.cc:407] Connection header received
0929 02:52:35.430008 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:35.430040 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:35.430541 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:35.430965 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:35.432436 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:35.434321 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:35.435548 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:35.436626 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:35.436810 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:35.440541 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:35.440555 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:35.440574 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:35.440631 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:35.444079 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:35.444744 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:35.444750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:35.444756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:35.444930 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:35.445312 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:35.445316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:35.445320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:35.445821 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:35.446129 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:35.446500 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 02:52:35.446897 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":346,"threads_started":1}
I20260929 02:52:35.454068 12564 raft_consensus.cc:348] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39893 } }
I20260929 02:52:35.454411 12564 raft_consensus.cc:374] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:35.454497 12564 raft_consensus.cc:729] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d910d2239fec4e16a5e24a921ef8bf6c, State: Initialized, Role: FOLLOWER
I20260929 02:52:35.455045 12564 consensus_queue.cc:260] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39893 } }
I20260929 02:52:35.455259 12564 raft_consensus.cc:388] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:35.455375 12564 raft_consensus.cc:482] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:35.455510 12564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:35.457692 12564 raft_consensus.cc:504] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39893 } }
I20260929 02:52:35.458163 12564 leader_election.cc:302] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d910d2239fec4e16a5e24a921ef8bf6c; no voters:
I20260929 02:52:35.458686 12564 leader_election.cc:288] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:35.458812 12569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:35.459443 12569 raft_consensus.cc:686] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [term 1 LEADER]: Becoming Leader. State: Replica: d910d2239fec4e16a5e24a921ef8bf6c, State: Running, Role: LEADER
I20260929 02:52:35.459996 12569 consensus_queue.cc:237] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39893 } }
I20260929 02:52:35.460639 12564 sys_catalog.cc:564] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:35.462406 12570 sys_catalog.cc:455] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39893 } } }
I20260929 02:52:35.462675 12570 sys_catalog.cc:458] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:35.464181 12574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:35.464027 12571 sys_catalog.cc:455] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d910d2239fec4e16a5e24a921ef8bf6c. Latest consensus state: current_term: 1 leader_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d910d2239fec4e16a5e24a921ef8bf6c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39893 } } }
I20260929 02:52:35.464351 12571 sys_catalog.cc:458] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:35.468729 12574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:35.477515 12574 catalog_manager.cc:1347] Generated new cluster ID: 2923e208f3644005a3e21921f4569407
I20260929 02:52:35.477625 12574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:35.485891 12574 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:35.487704 12574 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:35.502791 12574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c: Generated new TSK 0
I20260929 02:52:35.503939 12574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:35.553073 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:39893
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:35.859223 12590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:35.859483 12590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:35.859606 12590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:35.866744 12590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:35.866858 12590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:35.867017 12590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:35.876672 12590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:39893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:35.877894 12590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:35.879118 12590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:35.892287 12595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:35.892362 12596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:35.892314 12598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:35.892989 12590 server_base.cc:1034] running on GCE node
I20260929 02:52:35.893605 12590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:35.894519 12590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:35.895718 12590 hybrid_clock.cc:648] HybridClock initialized: now 1790650355895667 us; error 46 us; skew 500 ppm
I20260929 02:52:35.896052 12590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:35 dist-test-slave-1dkt krb5kdc[12485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:35.898916 12590 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:35.900030 12590 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:35.900125 12590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:35.900385 12590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:35.903204 12590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "8c94f05c192d41949cec374e71fb2b0b"
format_stamp: "Formatted at 2026-09-29 02:52:35 on dist-test-slave-1dkt"
server_key: "41fc4486214028401d02537b8fcd1871"
server_key_iv: "212429fa72aa0ff33ff742086366317b"
server_key_version: "encryptionkey@0"
I20260929 02:52:35.904000 12590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "8c94f05c192d41949cec374e71fb2b0b"
format_stamp: "Formatted at 2026-09-29 02:52:35 on dist-test-slave-1dkt"
server_key: "41fc4486214028401d02537b8fcd1871"
server_key_iv: "212429fa72aa0ff33ff742086366317b"
server_key_version: "encryptionkey@0"
I20260929 02:52:35.909061 12590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:52:35.912588 12604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:35.913904 12590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:35.914060 12590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "8c94f05c192d41949cec374e71fb2b0b"
format_stamp: "Formatted at 2026-09-29 02:52:35 on dist-test-slave-1dkt"
server_key: "41fc4486214028401d02537b8fcd1871"
server_key_iv: "212429fa72aa0ff33ff742086366317b"
server_key_version: "encryptionkey@0"
I20260929 02:52:35.914265 12590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:35.941811 12590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:35.945827 12590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:35.946095 12590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:35.947183 12590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:35.948560 12590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:35.948657 12590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:35.948757 12590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:35.948827 12590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:35.978036 12590 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:36093
I20260929 02:52:35.978103 12716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:36093 every 8 connection(s)
I20260929 02:52:35.978832 12590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:35.980871 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12590
I20260929 02:52:35.981101 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:35.981856 8420 external_mini_cluster.cc:1428] Setting key 6bd66eac0b6a026a37287951a5e7325b
Sep 29 02:52:35 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650355, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:35.998549 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:35.981240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:55391 (local address 127.8.57.62:39893)
0929 02:52:35.981660 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:35.981664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:35.982848 (+ 1184us) server_negotiation.cc:407] Connection header received
0929 02:52:35.984939 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:35.984945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:35.985028 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:35.985116 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:35.986586 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:35.987314 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:35.989233 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:35.989616 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:35.989723 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:35.993077 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:35.993081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:35.993085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:35.993118 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:35.995512 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:35.996268 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:35.996273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:35.996275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:35.996394 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:35.996798 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:35.996802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:35.996804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:35.997079 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:35.997209 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:35.997918 (+ 709us) server_negotiation.cc:299] Negotiation successful
0929 02:52:35.998156 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":213,"threads_started":1}
I20260929 02:52:35.999526 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:35.981910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:39893 (local address 127.8.57.1:55391)
0929 02:52:35.982707 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:35.982754 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:35.984684 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:35.985353 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:35.985361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:35.985862 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:35.986423 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:35.986439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:35.987461 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:35.987465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:35.989098 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:35.989110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:35.989821 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:35.989829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:35.990065 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:35.990675 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:35.990698 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:35.992833 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:35.995643 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:35.995652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:35.995669 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:35.996146 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:35.996502 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:35.996507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:35.996511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:35.996687 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:35.997200 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:35.997208 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:35.997545 (+ 337us) client_negotiation.cc:769] Sending connection context
0929 02:52:35.997801 (+ 256us) client_negotiation.cc:240] Negotiation successful
0929 02:52:35.998320 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":307,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:52:36.001230 12717 heartbeater.cc:347] Connected to a master server at 127.8.57.62:39893
I20260929 02:52:36.001528 12717 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:36.002146 12717 heartbeater.cc:511] Master 127.8.57.62:39893 requested a full tablet report, sending...
I20260929 02:52:36.004492 12529 ts_manager.cc:194] Registered new tserver with Master: 8c94f05c192d41949cec374e71fb2b0b (127.8.57.1:36093)
I20260929 02:52:36.006276 12529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:55391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:36.022907 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:39893
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:36.329967 12723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:36.330240 12723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:36.330335 12723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:36.337517 12723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:36.337621 12723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:36.337743 12723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:36.347262 12723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:39893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:36.348589 12723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:36.349799 12723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:36.361819 12729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:36.361845 12731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:36.362102 12728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:36.362459 12723 server_base.cc:1034] running on GCE node
I20260929 02:52:36.363039 12723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:36.363991 12723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:36.365183 12723 hybrid_clock.cc:648] HybridClock initialized: now 1790650356365130 us; error 51 us; skew 500 ppm
I20260929 02:52:36.365479 12723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:36 dist-test-slave-1dkt krb5kdc[12485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:36.368484 12723 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:36.369697 12723 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:36.369804 12723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:36.370126 12723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:36.373064 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "ba4b17bd5cff499d822b70da8c7448bd"
format_stamp: "Formatted at 2026-09-29 02:52:36 on dist-test-slave-1dkt"
server_key: "10a86fed27f48ad96aa21681087ca7fa"
server_key_iv: "ce6371a782e113dd9180f8caae10a856"
server_key_version: "encryptionkey@0"
I20260929 02:52:36.373857 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "ba4b17bd5cff499d822b70da8c7448bd"
format_stamp: "Formatted at 2026-09-29 02:52:36 on dist-test-slave-1dkt"
server_key: "10a86fed27f48ad96aa21681087ca7fa"
server_key_iv: "ce6371a782e113dd9180f8caae10a856"
server_key_version: "encryptionkey@0"
I20260929 02:52:36.379283 12723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:52:36.382990 12737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:36.384474 12723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:36.384634 12723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "ba4b17bd5cff499d822b70da8c7448bd"
format_stamp: "Formatted at 2026-09-29 02:52:36 on dist-test-slave-1dkt"
server_key: "10a86fed27f48ad96aa21681087ca7fa"
server_key_iv: "ce6371a782e113dd9180f8caae10a856"
server_key_version: "encryptionkey@0"
I20260929 02:52:36.384842 12723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:36.402573 12723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:36.406529 12723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:36.406769 12723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:36.407702 12723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:36.409032 12723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:36.409134 12723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:36.409237 12723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:36.409312 12723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:36.439368 12723 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:44587
I20260929 02:52:36.439428 12849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:44587 every 8 connection(s)
I20260929 02:52:36.440228 12723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:36.449116 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12723
I20260929 02:52:36.449285 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:36.449774 8420 external_mini_cluster.cc:1428] Setting key 3a8245c70ddea0f340883cab22568dd0
Sep 29 02:52:36 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:36.459844 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.442551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41785 (local address 127.8.57.62:39893)
0929 02:52:36.442741 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:36.442748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:36.443860 (+ 1112us) server_negotiation.cc:407] Connection header received
0929 02:52:36.445653 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:36.445657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:36.445738 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:36.445826 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:36.447253 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.448023 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.449610 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.450189 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.450317 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.453998 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:36.454004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:36.454009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:36.454052 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:36.456964 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:36.457687 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:36.457693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:36.457697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:36.457856 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:36.458205 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:36.458209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:36.458212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:36.458555 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:36.458697 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:36.459440 (+ 743us) server_negotiation.cc:299] Negotiation successful
0929 02:52:36.459612 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:52:36.461258 12852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.443035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:39893 (local address 127.8.57.2:41785)
0929 02:52:36.443700 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:36.443743 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:36.445392 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:36.446024 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:36.446032 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:36.446555 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:36.447117 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.447134 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.448175 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.448181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.449475 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.449486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.450399 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.450406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.450671 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.451309 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:36.451332 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:36.453765 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:36.457101 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:36.457109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:36.457124 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:36.457564 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:36.457949 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:36.457952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:36.457954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:36.458119 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:36.458661 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:36.458667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:36.459077 (+ 410us) client_negotiation.cc:769] Sending connection context
0929 02:52:36.459361 (+ 284us) client_negotiation.cc:240] Negotiation successful
0929 02:52:36.459938 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":262,"threads_started":1}
I20260929 02:52:36.462760 12850 heartbeater.cc:347] Connected to a master server at 127.8.57.62:39893
I20260929 02:52:36.463043 12850 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:36.463686 12850 heartbeater.cc:511] Master 127.8.57.62:39893 requested a full tablet report, sending...
I20260929 02:52:36.465176 12529 ts_manager.cc:194] Registered new tserver with Master: ba4b17bd5cff499d822b70da8c7448bd (127.8.57.2:44587)
I20260929 02:52:36.466331 12529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:41785
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:36.491097 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:39893
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:36.794809 12855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:36.795080 12855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:36.795215 12855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:36.802259 12855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:36.802371 12855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:36.802525 12855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:36.812075 12855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:39893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:36.813316 12855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:36.814606 12855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:36.826740 12860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:36.826786 12861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:36.826764 12863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:36.827446 12855 server_base.cc:1034] running on GCE node
I20260929 02:52:36.828003 12855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:36.828871 12855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:36.830046 12855 hybrid_clock.cc:648] HybridClock initialized: now 1790650356829993 us; error 49 us; skew 500 ppm
I20260929 02:52:36.830328 12855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:36 dist-test-slave-1dkt krb5kdc[12485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:36.833294 12855 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:36.834358 12855 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:36.834450 12855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:36.834745 12855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:36.837723 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "aa050547b66142e1a05b82f18451d4ea"
format_stamp: "Formatted at 2026-09-29 02:52:36 on dist-test-slave-1dkt"
server_key: "7fcadc0731200dab113ebdef21b6809e"
server_key_iv: "777b4ac162922f2e4dd9b695ef8bfba5"
server_key_version: "encryptionkey@0"
I20260929 02:52:36.838474 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "aa050547b66142e1a05b82f18451d4ea"
format_stamp: "Formatted at 2026-09-29 02:52:36 on dist-test-slave-1dkt"
server_key: "7fcadc0731200dab113ebdef21b6809e"
server_key_iv: "777b4ac162922f2e4dd9b695ef8bfba5"
server_key_version: "encryptionkey@0"
I20260929 02:52:36.843228 12855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 02:52:36.846578 12869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:36.847934 12855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:52:36.848084 12855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "aa050547b66142e1a05b82f18451d4ea"
format_stamp: "Formatted at 2026-09-29 02:52:36 on dist-test-slave-1dkt"
server_key: "7fcadc0731200dab113ebdef21b6809e"
server_key_iv: "777b4ac162922f2e4dd9b695ef8bfba5"
server_key_version: "encryptionkey@0"
I20260929 02:52:36.848289 12855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:36.860898 12855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:36.865027 12855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:36.865279 12855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:36.866273 12855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:36.867568 12855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:36.867671 12855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:36.867775 12855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:36.867841 12855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:36.897419 12855 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:33291
I20260929 02:52:36.897475 12981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:33291 every 8 connection(s)
I20260929 02:52:36.898227 12855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:36.906694 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 12855
I20260929 02:52:36.906867 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:36.907254 8420 external_mini_cluster.cc:1428] Setting key 55e0f62d1b0a27813b1497c50b9caab4
Sep 29 02:52:36 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650356, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:36.916733 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.900495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:33425 (local address 127.8.57.62:39893)
0929 02:52:36.900669 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:36.900673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:36.901775 (+ 1102us) server_negotiation.cc:407] Connection header received
0929 02:52:36.903512 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:36.903518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:36.903597 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:36.903683 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:36.905147 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.905868 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.907445 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.908006 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.908110 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.911533 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:36.911538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:36.911541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:36.911581 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:36.913926 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:36.914610 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:36.914613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:36.914616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:36.914730 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:36.915104 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:36.915109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:36.915112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:36.915380 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:36.915507 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:36.916187 (+ 680us) server_negotiation.cc:299] Negotiation successful
0929 02:52:36.916398 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:52:36.917670 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.900945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:39893 (local address 127.8.57.3:33425)
0929 02:52:36.901629 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:36.901671 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:36.903255 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:36.903941 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:36.903950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:36.904443 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:36.904984 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.905003 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.906029 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.906033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.907302 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.907314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.908216 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.908222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.908438 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.909258 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:36.909280 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:36.911343 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:36.914050 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:36.914086 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:36.914099 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:36.914496 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:36.914825 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:36.914831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:36.914835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:36.915005 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:36.915496 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:36.915503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:36.915821 (+ 318us) client_negotiation.cc:769] Sending connection context
0929 02:52:36.916098 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 02:52:36.916591 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":277,"threads_started":1}
I20260929 02:52:36.918958 12982 heartbeater.cc:347] Connected to a master server at 127.8.57.62:39893
I20260929 02:52:36.919231 12982 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:36.919862 12982 heartbeater.cc:511] Master 127.8.57.62:39893 requested a full tablet report, sending...
I20260929 02:52:36.921343 12528 ts_manager.cc:194] Registered new tserver with Master: aa050547b66142e1a05b82f18451d4ea (127.8.57.3:33291)
I20260929 02:52:36.922219 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:52:36.922319 12528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:33425
I20260929 02:52:36.939838 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.926528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.8.57.62:39893)
0929 02:52:36.926676 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:36.926680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:36.927146 (+ 466us) server_negotiation.cc:407] Connection header received
0929 02:52:36.927305 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:36.927309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:36.927368 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:36.927445 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:36.928871 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.929675 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.931118 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.931524 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.931620 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.933756 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:36.933763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:36.933768 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:36.933809 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:36.937005 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:36.937718 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:36.937725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:36.937728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:36.937882 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:36.938241 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:36.938246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:36.938250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:36.938701 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:36.938949 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:36.939266 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 02:52:36.939498 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:52:36.943675 12528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:36.950512 12528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:36.973029 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.961954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33291 (local address 127.0.0.1:33926)
0929 02:52:36.962912 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:36.962955 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:36.963155 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:36.965080 (+ 1925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:36.965088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:36.965120 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:36.965588 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.965600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.968026 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.968033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.970213 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.970227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.972064 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.972070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.972244 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.972298 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 02:52:36.972410 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 02:52:36.972604 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":1,"thread_start_us":542,"threads_started":1}
I20260929 02:52:36.973031 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.962245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.8.57.3:33291)
0929 02:52:36.964407 (+ 2162us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:36.964417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:36.964452 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:52:36.964553 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:36.964561 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:36.964832 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:36.965804 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:36.966125 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.967848 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.970390 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.971854 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.972029 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.972256 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:36.972463 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 02:52:36.972627 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1988,"thread_start_us":1834,"threads_started":1}
I20260929 02:52:36.975548 12996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.962565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:44587 (local address 127.0.0.1:48332)
0929 02:52:36.963730 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:36.963745 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:36.963844 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:36.965458 (+ 1614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:36.965463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:36.965508 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:36.965974 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.965984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.969290 (+ 3306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.969296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.971630 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.971641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.974729 (+ 3088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.974737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.974904 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.974938 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:52:36.975023 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 02:52:36.975180 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038,"mutex_wait_us":200,"thread_start_us":879,"threads_started":1}
I20260929 02:52:36.976626 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.963849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.8.57.2:44587)
0929 02:52:36.964847 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:36.964857 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:36.964885 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:52:36.964991 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:36.964999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:36.965230 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:36.966202 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:36.967573 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.969154 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.971777 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.974455 (+ 2678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.975864 (+ 1409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.976090 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:36.976168 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 02:52:36.976330 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":239,"threads_started":1}
I20260929 02:52:36.977327 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.963425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:36093 (local address 127.0.0.1:34222)
0929 02:52:36.970320 (+ 6895us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:36.970340 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:36.970456 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:36.971180 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:36.971185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:36.971212 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:36.971639 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.971651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.973478 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.973491 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.975125 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:36.975138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.976564 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.976570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:36.976730 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.976763 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:52:36.976844 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:52:36.976967 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6775,"thread_start_us":6641,"threads_started":1}
I20260929 02:52:36.977284 12917 tablet_service.cc:1467] Processing CreateTablet for tablet fe5df7fc5c1e44da845eb58097c560ec (DEFAULT_TABLE table=test-table [id=557d05bad9d54646abbb52078ed55764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:36.978782 12917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5df7fc5c1e44da845eb58097c560ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:36.980460 12785 tablet_service.cc:1467] Processing CreateTablet for tablet fe5df7fc5c1e44da845eb58097c560ec (DEFAULT_TABLE table=test-table [id=557d05bad9d54646abbb52078ed55764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:36.981782 12785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5df7fc5c1e44da845eb58097c560ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:36.984030 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:36.964213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.8.57.1:36093)
0929 02:52:36.968656 (+ 4443us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:36.968665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:36.970577 (+ 1912us) server_negotiation.cc:407] Connection header received
0929 02:52:36.970682 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:36.970689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:36.970934 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:36.971185 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:36.971809 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.973329 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.975295 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:36.976383 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:36.976515 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:36.976701 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:36.976894 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 02:52:36.977046 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4279,"thread_start_us":4182,"threads_started":1}
I20260929 02:52:36.987807 12652 tablet_service.cc:1467] Processing CreateTablet for tablet fe5df7fc5c1e44da845eb58097c560ec (DEFAULT_TABLE table=test-table [id=557d05bad9d54646abbb52078ed55764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:36.989161 12652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe5df7fc5c1e44da845eb58097c560ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:36.991834 13000 tablet_bootstrap.cc:492] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Bootstrap starting.
I20260929 02:52:36.994068 13001 tablet_bootstrap.cc:492] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Bootstrap starting.
I20260929 02:52:36.996248 13000 tablet_bootstrap.cc:654] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:36.997000 13001 tablet_bootstrap.cc:654] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:36.997449 13000 log.cc:824] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:36.997912 13001 log.cc:824] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:37.000504 13000 tablet_bootstrap.cc:492] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: No bootstrap required, opened a new log
I20260929 02:52:37.000674 13001 tablet_bootstrap.cc:492] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: No bootstrap required, opened a new log
I20260929 02:52:37.000777 13000 ts_tablet_manager.cc:1397] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:52:37.000911 13001 ts_tablet_manager.cc:1397] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:52:37.005432 13004 tablet_bootstrap.cc:492] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Bootstrap starting.
I20260929 02:52:37.008631 13004 tablet_bootstrap.cc:654] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:37.009330 12717 heartbeater.cc:503] Master 127.8.57.62:39893 was elected leader, sending a full tablet report...
I20260929 02:52:37.009881 13004 log.cc:824] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:37.011236 13000 raft_consensus.cc:348] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.011240 13001 raft_consensus.cc:348] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.011693 13001 raft_consensus.cc:374] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:37.011827 13001 raft_consensus.cc:729] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba4b17bd5cff499d822b70da8c7448bd, State: Initialized, Role: FOLLOWER
I20260929 02:52:37.011838 13000 raft_consensus.cc:374] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:37.011958 13000 raft_consensus.cc:729] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa050547b66142e1a05b82f18451d4ea, State: Initialized, Role: FOLLOWER
I20260929 02:52:37.012585 13001 consensus_queue.cc:260] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.012585 13000 consensus_queue.cc:260] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.016069 12850 heartbeater.cc:503] Master 127.8.57.62:39893 was elected leader, sending a full tablet report...
I20260929 02:52:37.016049 13001 ts_tablet_manager.cc:1428] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260929 02:52:37.017917 12982 heartbeater.cc:503] Master 127.8.57.62:39893 was elected leader, sending a full tablet report...
I20260929 02:52:37.018579 13000 ts_tablet_manager.cc:1428] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260929 02:52:37.019034 13004 tablet_bootstrap.cc:492] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: No bootstrap required, opened a new log
I20260929 02:52:37.019340 13004 ts_tablet_manager.cc:1397] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260929 02:52:37.026820 13004 raft_consensus.cc:348] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.027192 13004 raft_consensus.cc:374] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:37.027292 13004 raft_consensus.cc:729] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c94f05c192d41949cec374e71fb2b0b, State: Initialized, Role: FOLLOWER
I20260929 02:52:37.027842 13004 consensus_queue.cc:260] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.029470 13004 ts_tablet_manager.cc:1428] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Time spent starting tablet: real 0.010s user 0.002s sys 0.007s
I20260929 02:52:37.034458 13007 raft_consensus.cc:482] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:37.034729 13007 raft_consensus.cc:504] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.036301 13007 leader_election.cc:288] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa050547b66142e1a05b82f18451d4ea (127.8.57.3:33291), 8c94f05c192d41949cec374e71fb2b0b (127.8.57.1:36093)
I20260929 02:52:37.042891 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.036619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:48933 (local address 127.8.57.3:33291)
0929 02:52:37.036817 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:37.036823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:37.037335 (+ 512us) server_negotiation.cc:407] Connection header received
0929 02:52:37.037514 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:37.037520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:37.037607 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:37.037690 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:37.038291 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.039176 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.041098 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.041959 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.042052 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.042132 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:37.042491 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 02:52:37.042634 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:52:37.042973 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.037522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:53547 (local address 127.8.57.1:36093)
0929 02:52:37.037701 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:37.037706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:37.037962 (+ 256us) server_negotiation.cc:407] Connection header received
0929 02:52:37.038090 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:37.038095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:37.038167 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:37.038256 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:37.038799 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.039608 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.041338 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.042122 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.042244 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.042332 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:37.042610 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 02:52:37.042736 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:52:37.043314 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.036620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33291 (local address 127.8.57.2:48933)
0929 02:52:37.037234 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:37.037249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:37.037385 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:37.037783 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:37.037787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:37.037810 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:37.038153 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:37.038164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.039293 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.039299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:37.040949 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:37.040962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.042160 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.042167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:37.042334 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.042367 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:52:37.042444 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:52:37.042545 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":47,"thread_start_us":334,"threads_started":1}
I20260929 02:52:37.043352 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.037348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:36093 (local address 127.8.57.2:53547)
0929 02:52:37.037871 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:37.037884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:37.037983 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:37.038279 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:37.038283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:37.038304 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:37.038660 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:37.038672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.039736 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.039740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:37.041198 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:37.041212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.042317 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.042322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:37.042457 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.042487 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:52:37.042570 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 02:52:37.042687 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":2,"thread_start_us":243,"threads_started":1}
I20260929 02:52:37.044317 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5df7fc5c1e44da845eb58097c560ec" candidate_uuid: "ba4b17bd5cff499d822b70da8c7448bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa050547b66142e1a05b82f18451d4ea" is_pre_election: true
I20260929 02:52:37.044317 12672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5df7fc5c1e44da845eb58097c560ec" candidate_uuid: "ba4b17bd5cff499d822b70da8c7448bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c94f05c192d41949cec374e71fb2b0b" is_pre_election: true
I20260929 02:52:37.044862 12937 raft_consensus.cc:2454] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba4b17bd5cff499d822b70da8c7448bd in term 0.
I20260929 02:52:37.044864 12672 raft_consensus.cc:2454] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba4b17bd5cff499d822b70da8c7448bd in term 0.
I20260929 02:52:37.045512 12740 leader_election.cc:302] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa050547b66142e1a05b82f18451d4ea, ba4b17bd5cff499d822b70da8c7448bd; no voters:
I20260929 02:52:37.045856 13007 raft_consensus.cc:2781] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:37.045995 13007 raft_consensus.cc:482] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:37.046095 13007 raft_consensus.cc:3037] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:37.048235 13007 raft_consensus.cc:504] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.048768 13007 leader_election.cc:288] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [CANDIDATE]: Term 1 election: Requested vote from peers aa050547b66142e1a05b82f18451d4ea (127.8.57.3:33291), 8c94f05c192d41949cec374e71fb2b0b (127.8.57.1:36093)
I20260929 02:52:37.049177 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5df7fc5c1e44da845eb58097c560ec" candidate_uuid: "ba4b17bd5cff499d822b70da8c7448bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa050547b66142e1a05b82f18451d4ea"
I20260929 02:52:37.049263 12672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe5df7fc5c1e44da845eb58097c560ec" candidate_uuid: "ba4b17bd5cff499d822b70da8c7448bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c94f05c192d41949cec374e71fb2b0b"
I20260929 02:52:37.049378 12937 raft_consensus.cc:3037] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:37.049470 12672 raft_consensus.cc:3037] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:37.051592 12672 raft_consensus.cc:2454] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba4b17bd5cff499d822b70da8c7448bd in term 1.
I20260929 02:52:37.051612 12937 raft_consensus.cc:2454] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba4b17bd5cff499d822b70da8c7448bd in term 1.
I20260929 02:52:37.051944 12740 leader_election.cc:302] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa050547b66142e1a05b82f18451d4ea, ba4b17bd5cff499d822b70da8c7448bd; no voters:
I20260929 02:52:37.052219 13007 raft_consensus.cc:2781] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:37.052647 13007 raft_consensus.cc:686] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 1 LEADER]: Becoming Leader. State: Replica: ba4b17bd5cff499d822b70da8c7448bd, State: Running, Role: LEADER
I20260929 02:52:37.053186 13007 consensus_queue.cc:237] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } }
I20260929 02:52:37.057293 12529 catalog_manager.cc:5495] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd reported cstate change: term changed from 0 to 1, leader changed from <none> to ba4b17bd5cff499d822b70da8c7448bd (127.8.57.2). New cstate: current_term: 1 leader_uuid: "ba4b17bd5cff499d822b70da8c7448bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba4b17bd5cff499d822b70da8c7448bd" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 44587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:37.074451 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.069448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.8.57.2:44587)
0929 02:52:37.069632 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:37.069638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:37.069655 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:37.069738 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:37.069742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:37.069811 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:37.069931 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:37.070284 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.070966 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.072428 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.072831 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.072951 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.073427 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:37.073906 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:37.074052 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:37.074128 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 02:52:37.074207 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:52:37.082288 12937 raft_consensus.cc:1261] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 1 FOLLOWER]: Refusing update from remote peer ba4b17bd5cff499d822b70da8c7448bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:37.082294 12672 raft_consensus.cc:1261] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 1 FOLLOWER]: Refusing update from remote peer ba4b17bd5cff499d822b70da8c7448bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:37.083174 13011 consensus_queue.cc:1035] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c94f05c192d41949cec374e71fb2b0b" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:37.083500 13007 consensus_queue.cc:1035] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa050547b66142e1a05b82f18451d4ea" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:37.100150 13016 mvcc.cc:205] Tried to move back new op lower bound from 7334503862599266304 to 7334503862493446144. Current Snapshot: MvccSnapshot[applied={T|T < 7334503862599266304}]
I20260929 02:52:37.108263 12529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:37.108513 12529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:37.111601 12529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d910d2239fec4e16a5e24a921ef8bf6c: Sending DeleteTablet for 3 replicas of tablet fe5df7fc5c1e44da845eb58097c560ec
I20260929 02:52:37.112798 12917 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5df7fc5c1e44da845eb58097c560ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:37 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:33926
I20260929 02:52:37.112835 12785 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5df7fc5c1e44da845eb58097c560ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:37 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:48332
I20260929 02:52:37.113727 12652 tablet_service.cc:1514] Processing DeleteTablet for tablet fe5df7fc5c1e44da845eb58097c560ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:37 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:34222
I20260929 02:52:37.113916 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12590
I20260929 02:52:37.123769 13026 tablet_replica.cc:330] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: stopping tablet replica
I20260929 02:52:37.124444 13024 tablet_replica.cc:330] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: stopping tablet replica
I20260929 02:52:37.125324 13026 raft_consensus.cc:2229] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:37.125851 13026 raft_consensus.cc:2258] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:37.125918 13024 raft_consensus.cc:2229] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:37.126770 13024 raft_consensus.cc:2258] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:37.127794 13025 tablet_replica.cc:330] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: stopping tablet replica
I20260929 02:52:37.128288 13025 raft_consensus.cc:2229] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:37.128957 13025 raft_consensus.cc:2258] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:37.129668 13024 ts_tablet_manager.cc:1905] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 02:52:37 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:37.138375 13024 ts_tablet_manager.cc:1918] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:37.138726 13024 log.cc:1196] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/fe5df7fc5c1e44da845eb58097c560ec
I20260929 02:52:37.139605 13025 ts_tablet_manager.cc:1905] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:37.141312 13026 ts_tablet_manager.cc:1905] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:37.141832 13024 ts_tablet_manager.cc:1939] T fe5df7fc5c1e44da845eb58097c560ec P aa050547b66142e1a05b82f18451d4ea: Deleting consensus metadata
I20260929 02:52:37.144598 12515 catalog_manager.cc:4841] TS aa050547b66142e1a05b82f18451d4ea (127.8.57.3:33291): tablet fe5df7fc5c1e44da845eb58097c560ec (table test-table [id=557d05bad9d54646abbb52078ed55764]) successfully deleted
I20260929 02:52:37.146631 13025 ts_tablet_manager.cc:1918] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:37.146795 13025 log.cc:1196] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/fe5df7fc5c1e44da845eb58097c560ec
I20260929 02:52:37.147308 13025 ts_tablet_manager.cc:1939] T fe5df7fc5c1e44da845eb58097c560ec P ba4b17bd5cff499d822b70da8c7448bd: Deleting consensus metadata
I20260929 02:52:37.148470 13026 ts_tablet_manager.cc:1918] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:37.148533 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.115477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.8.57.1:36093)
0929 02:52:37.115634 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:37.115640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:37.124702 (+ 9062us) server_negotiation.cc:407] Connection header received
0929 02:52:37.124882 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:37.124888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:37.125020 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:37.125132 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:37.126836 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.128359 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.130086 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.130805 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.130975 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.139349 (+ 8374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:37.139396 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:37.139416 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:37.139504 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:37.145206 (+ 5702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:37.146130 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:37.146140 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:37.146144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:37.146297 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:37.146705 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:37.146710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:37.146714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:37.147231 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:37.147509 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:37.147941 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 02:52:37.148152 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 02:52:37.148674 13026 log.cc:1196] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/fe5df7fc5c1e44da845eb58097c560ec
I20260929 02:52:37.149091 12516 catalog_manager.cc:4841] TS ba4b17bd5cff499d822b70da8c7448bd (127.8.57.2:44587): tablet fe5df7fc5c1e44da845eb58097c560ec (table test-table [id=557d05bad9d54646abbb52078ed55764]) successfully deleted
I20260929 02:52:37.149217 13026 ts_tablet_manager.cc:1939] T fe5df7fc5c1e44da845eb58097c560ec P 8c94f05c192d41949cec374e71fb2b0b: Deleting consensus metadata
I20260929 02:52:37.149560 12712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:37.237056 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12590
I20260929 02:52:37.237195 12515 catalog_manager.cc:4841] TS 8c94f05c192d41949cec374e71fb2b0b (127.8.57.1:36093): tablet fe5df7fc5c1e44da845eb58097c560ec (table test-table [id=557d05bad9d54646abbb52078ed55764]) successfully deleted
I20260929 02:52:37.250037 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12723
Sep 29 02:52:37 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:37.265869 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.250788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.8.57.2:44587)
0929 02:52:37.250983 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:37.250988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:37.251029 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:52:37.251196 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:37.251202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:37.251289 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:37.251396 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:37.252623 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.253891 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.255053 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.255732 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.255857 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.259898 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:37.259919 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:37.259936 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:37.259977 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:37.262881 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:37.263693 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:37.263701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:37.263703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:37.263844 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:37.264475 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:37.264479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:37.264481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:37.264858 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:37.265117 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:37.265375 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 02:52:37.265569 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:52:37.266539 12845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:37.376865 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12723
I20260929 02:52:37.390556 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12855
Sep 29 02:52:37 dist-test-slave-1dkt krb5kdc[12485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:37.404652 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:37.391213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.8.57.3:33291)
0929 02:52:37.391380 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:37.391386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:37.391452 (+ 66us) server_negotiation.cc:407] Connection header received
0929 02:52:37.391619 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:37.391625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:37.391693 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:37.391771 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:37.393011 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.393720 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.394906 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:37.395294 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:37.395400 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:37.399315 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:37.399328 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:37.399343 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:37.399382 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:37.402090 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:37.402696 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:37.402704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:37.402706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:37.402833 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:37.403168 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:37.403172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:37.403174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:37.403542 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:37.403799 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:37.404136 (+ 337us) server_negotiation.cc:299] Negotiation successful
0929 02:52:37.404341 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:52:37.405242 12977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:37.506153 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12855
I20260929 02:52:37.518138 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 12498
I20260929 02:52:37.519240 12559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:37.626022 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 12498
2026-09-29T02:52:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3783 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T02:52:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:37Z Disabled control of system clock
I20260929 02:52:37.683293 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:45787
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:45787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:52:37.994920 13035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:37.995210 13035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:37.995333 13035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:38.002643 13035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:38.002753 13035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:38.002815 13035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:38.002902 13035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:38.002964 13035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:38.012420 13035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:45787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:45787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:38.013710 13035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:38.015537 13035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:38.026643 13040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:38.026685 13043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:38.026605 13041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:38.027575 13035 server_base.cc:1034] running on GCE node
I20260929 02:52:38.028380 13035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:38.029707 13035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:38.030900 13035 hybrid_clock.cc:648] HybridClock initialized: now 1790650358030849 us; error 46 us; skew 500 ppm
I20260929 02:52:38.031217 13035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:38.033382 13035 webserver.cc:460] Webserver started at http://127.8.57.62:44393/ using document root <none> and password file <none>
I20260929 02:52:38.034029 13035 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:38.034126 13035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:38.034407 13035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:38.037243 13035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "ec2b9251603340fabca5a82ae73dc9a4"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "5658d6326c8baa0a3b6bc7d44b30f091"
server_key_iv: "e6ef40a69e2425663aede4e99deeceaa"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.037984 13035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "ec2b9251603340fabca5a82ae73dc9a4"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "5658d6326c8baa0a3b6bc7d44b30f091"
server_key_iv: "e6ef40a69e2425663aede4e99deeceaa"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.042893 13035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:52:38.046125 13049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.047338 13035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 02:52:38.047497 13035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "ec2b9251603340fabca5a82ae73dc9a4"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "5658d6326c8baa0a3b6bc7d44b30f091"
server_key_iv: "e6ef40a69e2425663aede4e99deeceaa"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.047685 13035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:38.052995 13035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:38.053721 13035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:38.053979 13035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:38.069919 13035 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:45787
I20260929 02:52:38.069937 13100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:45787 every 8 connection(s)
I20260929 02:52:38.071152 13035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:38.074723 13101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:38.079461 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13035
I20260929 02:52:38.079666 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:38.080470 8420 external_mini_cluster.cc:1428] Setting key 7c72fc1846a180201141edfe611adabb
I20260929 02:52:38.085764 13101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4: Bootstrap starting.
I20260929 02:52:38.088395 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:38.082434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.8.57.62:45787)
0929 02:52:38.083400 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:38.083411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:38.083455 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:52:38.085114 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:38.085136 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:38.085559 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:38.085903 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:38.086881 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:38.086888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:38.086900 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:38.086915 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:38.086968 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:38.086983 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:38.087143 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:38.087237 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 02:52:38.087477 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":326,"threads_started":1}
I20260929 02:52:38.089108 13101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:38.090088 13101 log.cc:824] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:38.092783 13101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4: No bootstrap required, opened a new log
I20260929 02:52:38.099725 13101 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2b9251603340fabca5a82ae73dc9a4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45787 } }
I20260929 02:52:38.100185 13101 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:38.100296 13101 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2b9251603340fabca5a82ae73dc9a4, State: Initialized, Role: FOLLOWER
I20260929 02:52:38.100875 13101 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2b9251603340fabca5a82ae73dc9a4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45787 } }
I20260929 02:52:38.101095 13101 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:38.101221 13101 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:38.101354 13101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:38.103410 13101 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2b9251603340fabca5a82ae73dc9a4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45787 } }
I20260929 02:52:38.103900 13101 leader_election.cc:302] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec2b9251603340fabca5a82ae73dc9a4; no voters:
I20260929 02:52:38.104453 13101 leader_election.cc:288] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:38.104606 13106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:38.105237 13106 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [term 1 LEADER]: Becoming Leader. State: Replica: ec2b9251603340fabca5a82ae73dc9a4, State: Running, Role: LEADER
I20260929 02:52:38.105782 13106 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2b9251603340fabca5a82ae73dc9a4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45787 } }
I20260929 02:52:38.106531 13101 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:38.108918 13107 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec2b9251603340fabca5a82ae73dc9a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2b9251603340fabca5a82ae73dc9a4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45787 } } }
I20260929 02:52:38.108940 13108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec2b9251603340fabca5a82ae73dc9a4. Latest consensus state: current_term: 1 leader_uuid: "ec2b9251603340fabca5a82ae73dc9a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2b9251603340fabca5a82ae73dc9a4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45787 } } }
I20260929 02:52:38.109444 13107 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:38.109442 13108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:38.110970 13115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:38.115401 13115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:38.123068 13115 catalog_manager.cc:1347] Generated new cluster ID: 7c1d4148432b44a6994ddfe3cbaf0d51
I20260929 02:52:38.123193 13115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:38.141315 13115 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:38.141860 13115 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:38.148502 13115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4: Generated new TSK 0
I20260929 02:52:38.149572 13115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:52:38.161633 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45787
--builtin_ntp_servers=127.8.57.20:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:52:38.471623 13125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:38.471946 13125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:38.472082 13125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:38.479336 13125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:38.479439 13125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:38.479590 13125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:38.488903 13125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:38.490165 13125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:38.491932 13125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:38.504277 13130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:38.504326 13131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:38.504304 13133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:38.505236 13125 server_base.cc:1034] running on GCE node
I20260929 02:52:38.505777 13125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:38.506767 13125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:38.507978 13125 hybrid_clock.cc:648] HybridClock initialized: now 1790650358507934 us; error 37 us; skew 500 ppm
I20260929 02:52:38.508262 13125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:38.515534 13125 webserver.cc:460] Webserver started at http://127.8.57.1:45231/ using document root <none> and password file <none>
I20260929 02:52:38.516281 13125 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:38.516376 13125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:38.516644 13125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:38.519598 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "84202331fa604302a9642487069195ca"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "82aa70b5cbf9166f6c75e4d9732b0bcf"
server_key_iv: "42e6072f65827f42772b147c0a1dcf7b"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.520390 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "84202331fa604302a9642487069195ca"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "82aa70b5cbf9166f6c75e4d9732b0bcf"
server_key_iv: "42e6072f65827f42772b147c0a1dcf7b"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.525349 13125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 02:52:38.528676 13139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.529951 13125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:38.530076 13125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "84202331fa604302a9642487069195ca"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "82aa70b5cbf9166f6c75e4d9732b0bcf"
server_key_iv: "42e6072f65827f42772b147c0a1dcf7b"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.530256 13125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:38.535485 13125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:38.536212 13125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:38.536417 13125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:38.537220 13125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:38.538611 13125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:38.538689 13125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.538765 13125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:38.538815 13125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.567417 13125 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:43339
I20260929 02:52:38.567487 13251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:43339 every 8 connection(s)
I20260929 02:52:38.568725 13125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:38.576344 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:38.571173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:44513 (local address 127.8.57.62:45787)
0929 02:52:38.571335 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:38.571339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:38.572635 (+ 1296us) server_negotiation.cc:407] Connection header received
0929 02:52:38.574249 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:38.574255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:38.574318 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:38.574401 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:38.575641 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:38.575647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:38.575651 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:38.575669 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:38.575700 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:38.575711 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:38.575815 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:38.576019 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 02:52:38.576089 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:52:38.577257 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:38.571598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:45787 (local address 127.8.57.1:44513)
0929 02:52:38.572494 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:38.572528 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:38.573996 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:38.574630 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:38.574638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:38.574730 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:38.575347 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:38.575349 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:38.575388 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:38.575475 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:38.575794 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:38.575804 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:38.575811 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:52:38.576097 (+ 286us) client_negotiation.cc:240] Negotiation successful
0929 02:52:38.576344 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":288,"threads_started":1}
I20260929 02:52:38.578730 13252 heartbeater.cc:347] Connected to a master server at 127.8.57.62:45787
I20260929 02:52:38.578830 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13125
I20260929 02:52:38.579011 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:38.579039 13252 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:38.579594 13252 heartbeater.cc:511] Master 127.8.57.62:45787 requested a full tablet report, sending...
I20260929 02:52:38.579622 8420 external_mini_cluster.cc:1428] Setting key a8805a9fe1d33c45465fcef3590121e5
I20260929 02:52:38.581670 13065 ts_manager.cc:194] Registered new tserver with Master: 84202331fa604302a9642487069195ca (127.8.57.1:43339)
I20260929 02:52:38.582257 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45787
--builtin_ntp_servers=127.8.57.20:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:52:38.890867 13255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:38.891157 13255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:38.891276 13255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:38.898402 13255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:38.898499 13255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:38.898641 13255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:38.907722 13255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:38.908964 13255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:38.910722 13255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:38.922971 13261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:38.923075 13260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:38.923058 13263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:38.923617 13255 server_base.cc:1034] running on GCE node
I20260929 02:52:38.924191 13255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:38.925082 13255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:38.926266 13255 hybrid_clock.cc:648] HybridClock initialized: now 1790650358926221 us; error 37 us; skew 500 ppm
I20260929 02:52:38.926568 13255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:38.928890 13255 webserver.cc:460] Webserver started at http://127.8.57.2:44523/ using document root <none> and password file <none>
I20260929 02:52:38.929576 13255 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:38.929667 13255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:38.929960 13255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:38.932811 13255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "adc2e9f879454921a57a92e56e00a7e9"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "7e5d97dff2b807df0f4a41bc00111179"
server_key_iv: "8cd4395b6e57c9be8ebb173b6740bbc1"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.933593 13255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "adc2e9f879454921a57a92e56e00a7e9"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "7e5d97dff2b807df0f4a41bc00111179"
server_key_iv: "8cd4395b6e57c9be8ebb173b6740bbc1"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.938539 13255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:52:38.941804 13269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.943050 13255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:38.943178 13255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "adc2e9f879454921a57a92e56e00a7e9"
format_stamp: "Formatted at 2026-09-29 02:52:38 on dist-test-slave-1dkt"
server_key: "7e5d97dff2b807df0f4a41bc00111179"
server_key_iv: "8cd4395b6e57c9be8ebb173b6740bbc1"
server_key_version: "encryptionkey@0"
I20260929 02:52:38.943377 13255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:38.948951 13255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:38.949681 13255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:38.949918 13255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:38.950773 13255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:38.952198 13255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:38.952284 13255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.952390 13255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:38.952457 13255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:38.981192 13255 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:41689
I20260929 02:52:38.981232 13381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:41689 every 8 connection(s)
I20260929 02:52:38.982448 13255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:38.988720 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13255
I20260929 02:52:38.988900 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:38.989459 8420 external_mini_cluster.cc:1428] Setting key 5477bdf5d8922df525606b962a3b3b53
I20260929 02:52:38.990931 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:38.984905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60895 (local address 127.8.57.62:45787)
0929 02:52:38.985082 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:38.985087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:38.986477 (+ 1390us) server_negotiation.cc:407] Connection header received
0929 02:52:38.988533 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:38.988539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:38.988600 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:38.988691 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:38.990177 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:38.990183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:38.990187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:38.990209 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:38.990249 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:38.990263 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:38.990352 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:38.990571 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 02:52:38.990632 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:52:38.992007 13384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:38.985401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:45787 (local address 127.8.57.2:60895)
0929 02:52:38.986326 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:38.986363 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:38.988123 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:38.988973 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:38.988985 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:38.989120 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:38.989809 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:38.989813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:38.989866 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:38.989983 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:38.990350 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:38.990359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:38.990367 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:52:38.990626 (+ 259us) client_negotiation.cc:240] Negotiation successful
0929 02:52:38.990831 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":279,"threads_started":1}
I20260929 02:52:38.992580 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45787
--builtin_ntp_servers=127.8.57.20:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 02:52:38.993332 13382 heartbeater.cc:347] Connected to a master server at 127.8.57.62:45787
I20260929 02:52:38.993625 13382 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:38.994185 13382 heartbeater.cc:511] Master 127.8.57.62:45787 requested a full tablet report, sending...
I20260929 02:52:38.995555 13065 ts_manager.cc:194] Registered new tserver with Master: adc2e9f879454921a57a92e56e00a7e9 (127.8.57.2:41689)
W20260929 02:52:39.303834 13385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:39.304179 13385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:39.304307 13385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:39.311748 13385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:39.311857 13385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:39.312024 13385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:39.321265 13385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:39.322489 13385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:39.324723 13385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:39.338125 13390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:39.338090 13391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:39.338165 13393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:39.338946 13385 server_base.cc:1034] running on GCE node
I20260929 02:52:39.339490 13385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:39.340420 13385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:39.341606 13385 hybrid_clock.cc:648] HybridClock initialized: now 1790650359341560 us; error 41 us; skew 500 ppm
I20260929 02:52:39.341892 13385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:39.343972 13385 webserver.cc:460] Webserver started at http://127.8.57.3:34803/ using document root <none> and password file <none>
I20260929 02:52:39.344621 13385 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:39.344713 13385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:39.345002 13385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:39.347852 13385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "fbca2563b634493fbbb12ab764ad89dd"
format_stamp: "Formatted at 2026-09-29 02:52:39 on dist-test-slave-1dkt"
server_key: "0a9bcb90f833874ffff6a9e988275a7c"
server_key_iv: "e9531ac05b56f249f2f62dfaa6b3173f"
server_key_version: "encryptionkey@0"
I20260929 02:52:39.348641 13385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "fbca2563b634493fbbb12ab764ad89dd"
format_stamp: "Formatted at 2026-09-29 02:52:39 on dist-test-slave-1dkt"
server_key: "0a9bcb90f833874ffff6a9e988275a7c"
server_key_iv: "e9531ac05b56f249f2f62dfaa6b3173f"
server_key_version: "encryptionkey@0"
I20260929 02:52:39.353554 13385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:39.356838 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:39.358114 13385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:39.358255 13385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "fbca2563b634493fbbb12ab764ad89dd"
format_stamp: "Formatted at 2026-09-29 02:52:39 on dist-test-slave-1dkt"
server_key: "0a9bcb90f833874ffff6a9e988275a7c"
server_key_iv: "e9531ac05b56f249f2f62dfaa6b3173f"
server_key_version: "encryptionkey@0"
I20260929 02:52:39.358460 13385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:39.363802 13385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:39.364609 13385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:39.364847 13385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:39.365680 13385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:39.366993 13385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:39.367084 13385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:39.367185 13385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:39.367251 13385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:39.395920 13385 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:36481
I20260929 02:52:39.395961 13511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:36481 every 8 connection(s)
I20260929 02:52:39.397183 13385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:39.399268 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13385
I20260929 02:52:39.399407 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:39.399957 8420 external_mini_cluster.cc:1428] Setting key 20b1e1bad219ad65d5dc83c3a20d7056
I20260929 02:52:39.405277 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.399639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:45899 (local address 127.8.57.62:45787)
0929 02:52:39.399897 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.399904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.401488 (+ 1584us) server_negotiation.cc:407] Connection header received
0929 02:52:39.403449 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.403452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.403512 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.403590 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.404755 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.404759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.404762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.404777 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.404807 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.404818 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.404910 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.405034 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.405081 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:52:39.406232 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.400246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:45787 (local address 127.8.57.3:45899)
0929 02:52:39.401342 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:39.401391 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:39.403207 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:39.403791 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:39.403800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:39.403930 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:39.404485 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:39.404487 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:39.404522 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:39.404618 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:39.404886 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:39.404891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:39.404896 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:52:39.405093 (+ 197us) client_negotiation.cc:240] Negotiation successful
0929 02:52:39.405290 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":300,"threads_started":1}
I20260929 02:52:39.407475 13512 heartbeater.cc:347] Connected to a master server at 127.8.57.62:45787
I20260929 02:52:39.407775 13512 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:39.408470 13512 heartbeater.cc:511] Master 127.8.57.62:45787 requested a full tablet report, sending...
I20260929 02:52:39.409885 13065 ts_manager.cc:194] Registered new tserver with Master: fbca2563b634493fbbb12ab764ad89dd (127.8.57.3:36481)
I20260929 02:52:39.415207 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:52:39.421754 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.419360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.8.57.62:45787)
0929 02:52:39.419509 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.419513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.419972 (+ 459us) server_negotiation.cc:407] Connection header received
0929 02:52:39.420153 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.420158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.420207 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.420282 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.421148 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.421154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.421159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.421177 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.421206 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.421217 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.421317 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.421409 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.421466 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 02:52:39.425356 13065 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:39.431962 13065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:39.459117 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.443492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.8.57.1:43339)
0929 02:52:39.455368 (+ 11876us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.455378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.455913 (+ 535us) server_negotiation.cc:407] Connection header received
0929 02:52:39.455984 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.456001 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.456252 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.456969 (+ 717us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.458229 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.458235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.458244 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.458263 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.458296 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.458308 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.458615 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.458685 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.458788 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11727,"thread_start_us":11610,"threads_started":1}
I20260929 02:52:39.459262 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.443153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:36481 (local address 127.0.0.1:48628)
0929 02:52:39.454655 (+ 11502us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:39.454703 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:39.454863 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:39.457279 (+ 2416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:39.457284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:39.457306 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:39.457968 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:39.457970 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:39.457985 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:39.458024 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:39.458410 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:39.458420 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:39.458428 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:52:39.458586 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 02:52:39.458687 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11176,"thread_start_us":12312,"threads_started":1}
I20260929 02:52:39.459244 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.443331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48654)
0929 02:52:39.455663 (+ 12332us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:39.455679 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:39.455774 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:39.456448 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:39.456456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:39.456554 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:39.457267 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:39.457271 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:39.457299 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:39.457393 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:39.458421 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:39.458426 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:39.458431 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:52:39.458624 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 02:52:39.458699 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12223,"mutex_wait_us":22,"thread_start_us":12060,"threads_started":1}
I20260929 02:52:39.460605 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.443271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.8.57.3:36481)
0929 02:52:39.456505 (+ 13234us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.456512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.456538 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:52:39.456599 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.456605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.456746 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.456874 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.458171 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.458180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.458194 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.458214 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.458279 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.458301 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.459921 (+ 1620us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.459992 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.460097 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13094,"thread_start_us":12954,"threads_started":1}
I20260929 02:52:39.461539 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.457462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:41689 (local address 127.0.0.1:56510)
0929 02:52:39.458273 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:39.458288 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:39.458382 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:39.459740 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:39.459744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:39.459769 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:39.460466 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:39.460469 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:39.460484 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:39.460530 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:39.460961 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:39.460964 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:39.460968 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 02:52:39.461085 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 02:52:39.461179 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":410,"threads_started":1}
I20260929 02:52:39.461650 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.456603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.8.57.2:41689)
0929 02:52:39.457183 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.457191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.458393 (+ 1202us) server_negotiation.cc:407] Connection header received
0929 02:52:39.458476 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.458483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.458691 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.458823 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.460659 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.460667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.460680 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.460707 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.460767 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.460786 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.460918 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.461193 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.461293 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":534,"threads_started":1}
I20260929 02:52:39.463455 13187 tablet_service.cc:1467] Processing CreateTablet for tablet a33510b67f754fa78bdf65b6c02a9949 (DEFAULT_TABLE table=test-table [id=be04ba60b0a24762b9f57ca52abbc7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:39.463690 13447 tablet_service.cc:1467] Processing CreateTablet for tablet a33510b67f754fa78bdf65b6c02a9949 (DEFAULT_TABLE table=test-table [id=be04ba60b0a24762b9f57ca52abbc7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:39.464771 13187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a33510b67f754fa78bdf65b6c02a9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:39.464897 13447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a33510b67f754fa78bdf65b6c02a9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:39.467700 13317 tablet_service.cc:1467] Processing CreateTablet for tablet a33510b67f754fa78bdf65b6c02a9949 (DEFAULT_TABLE table=test-table [id=be04ba60b0a24762b9f57ca52abbc7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:39.469061 13317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a33510b67f754fa78bdf65b6c02a9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:39.475835 13530 tablet_bootstrap.cc:492] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Bootstrap starting.
I20260929 02:52:39.478911 13530 tablet_bootstrap.cc:654] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:39.479723 13531 tablet_bootstrap.cc:492] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Bootstrap starting.
I20260929 02:52:39.480146 13530 log.cc:824] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:39.483095 13531 tablet_bootstrap.cc:654] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:39.483112 13533 tablet_bootstrap.cc:492] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Bootstrap starting.
I20260929 02:52:39.483152 13530 tablet_bootstrap.cc:492] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: No bootstrap required, opened a new log
I20260929 02:52:39.483436 13530 ts_tablet_manager.cc:1397] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260929 02:52:39.484445 13531 log.cc:824] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:39.486593 13533 tablet_bootstrap.cc:654] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:39.487337 13531 tablet_bootstrap.cc:492] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: No bootstrap required, opened a new log
I20260929 02:52:39.487607 13531 ts_tablet_manager.cc:1397] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 02:52:39.487746 13533 log.cc:824] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:39.490587 13533 tablet_bootstrap.cc:492] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: No bootstrap required, opened a new log
I20260929 02:52:39.490829 13533 ts_tablet_manager.cc:1397] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 02:52:39.490860 13530 raft_consensus.cc:348] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.491211 13530 raft_consensus.cc:374] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:39.491300 13530 raft_consensus.cc:729] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbca2563b634493fbbb12ab764ad89dd, State: Initialized, Role: FOLLOWER
I20260929 02:52:39.491829 13530 consensus_queue.cc:260] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.496599 13512 heartbeater.cc:503] Master 127.8.57.62:45787 was elected leader, sending a full tablet report...
I20260929 02:52:39.497133 13530 ts_tablet_manager.cc:1428] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260929 02:52:39.498195 13531 raft_consensus.cc:348] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.498632 13531 raft_consensus.cc:374] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:39.498728 13531 raft_consensus.cc:729] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84202331fa604302a9642487069195ca, State: Initialized, Role: FOLLOWER
I20260929 02:52:39.499342 13531 consensus_queue.cc:260] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.500422 13252 heartbeater.cc:503] Master 127.8.57.62:45787 was elected leader, sending a full tablet report...
I20260929 02:52:39.501065 13531 ts_tablet_manager.cc:1428] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 02:52:39.501844 13533 raft_consensus.cc:348] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.502213 13533 raft_consensus.cc:374] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:39.502305 13533 raft_consensus.cc:729] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc2e9f879454921a57a92e56e00a7e9, State: Initialized, Role: FOLLOWER
I20260929 02:52:39.502941 13533 consensus_queue.cc:260] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.503957 13382 heartbeater.cc:503] Master 127.8.57.62:45787 was elected leader, sending a full tablet report...
I20260929 02:52:39.504639 13533 ts_tablet_manager.cc:1428] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260929 02:52:39.570724 13253 tablet.cc:2369] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:52:39.648702 13513 tablet.cc:2369] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:52:39.734346 13383 tablet.cc:2369] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:39.787961 13538 raft_consensus.cc:482] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:39.788264 13538 raft_consensus.cc:504] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.789829 13538 leader_election.cc:288] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbca2563b634493fbbb12ab764ad89dd (127.8.57.3:36481), 84202331fa604302a9642487069195ca (127.8.57.1:43339)
I20260929 02:52:39.792932 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.790313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:58611 (local address 127.8.57.3:36481)
0929 02:52:39.790504 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.790511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.791158 (+ 647us) server_negotiation.cc:407] Connection header received
0929 02:52:39.791304 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.791308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.791356 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.791430 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.792339 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.792345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.792349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.792366 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.792394 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.792405 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.792509 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.792632 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.792692 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:52:39.793084 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.790389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41295 (local address 127.8.57.1:43339)
0929 02:52:39.790574 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.790581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.791247 (+ 666us) server_negotiation.cc:407] Connection header received
0929 02:52:39.791367 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.791372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.791420 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.791506 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.792508 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.792512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.792515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.792530 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.792560 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.792570 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.792675 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.792764 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.792822 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:52:39.793183 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.790218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:36481 (local address 127.8.57.2:58611)
0929 02:52:39.791053 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:39.791068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:39.791186 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:39.791467 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:39.791472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:39.791493 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:39.792158 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:39.792160 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:39.792173 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:39.792209 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:39.792481 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:39.792486 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:39.792491 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:52:39.792643 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 02:52:39.792688 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":542,"threads_started":1}
I20260929 02:52:39.793275 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.790264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.8.57.2:41295)
0929 02:52:39.791142 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:39.791155 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:39.791255 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:39.791534 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:39.791538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:39.791564 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:39.792302 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:52:39.792305 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:39.792317 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:52:39.792351 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:39.792660 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:39.792665 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:39.792670 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:52:39.792798 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 02:52:39.792843 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":456,"threads_started":1}
I20260929 02:52:39.794284 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33510b67f754fa78bdf65b6c02a9949" candidate_uuid: "adc2e9f879454921a57a92e56e00a7e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbca2563b634493fbbb12ab764ad89dd" is_pre_election: true
I20260929 02:52:39.794286 13207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33510b67f754fa78bdf65b6c02a9949" candidate_uuid: "adc2e9f879454921a57a92e56e00a7e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84202331fa604302a9642487069195ca" is_pre_election: true
I20260929 02:52:39.794826 13467 raft_consensus.cc:2454] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc2e9f879454921a57a92e56e00a7e9 in term 0.
I20260929 02:52:39.794826 13207 raft_consensus.cc:2454] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc2e9f879454921a57a92e56e00a7e9 in term 0.
I20260929 02:52:39.795464 13272 leader_election.cc:302] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84202331fa604302a9642487069195ca, adc2e9f879454921a57a92e56e00a7e9; no voters:
I20260929 02:52:39.795794 13538 raft_consensus.cc:2781] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:39.795964 13538 raft_consensus.cc:482] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:39.796059 13538 raft_consensus.cc:3037] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:39.798276 13538 raft_consensus.cc:504] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.798813 13538 leader_election.cc:288] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [CANDIDATE]: Term 1 election: Requested vote from peers fbca2563b634493fbbb12ab764ad89dd (127.8.57.3:36481), 84202331fa604302a9642487069195ca (127.8.57.1:43339)
I20260929 02:52:39.799204 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33510b67f754fa78bdf65b6c02a9949" candidate_uuid: "adc2e9f879454921a57a92e56e00a7e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbca2563b634493fbbb12ab764ad89dd"
I20260929 02:52:39.799229 13207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33510b67f754fa78bdf65b6c02a9949" candidate_uuid: "adc2e9f879454921a57a92e56e00a7e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84202331fa604302a9642487069195ca"
I20260929 02:52:39.799376 13467 raft_consensus.cc:3037] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:39.799389 13207 raft_consensus.cc:3037] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:39.801489 13467 raft_consensus.cc:2454] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc2e9f879454921a57a92e56e00a7e9 in term 1.
I20260929 02:52:39.801501 13207 raft_consensus.cc:2454] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc2e9f879454921a57a92e56e00a7e9 in term 1.
I20260929 02:52:39.801860 13272 leader_election.cc:302] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84202331fa604302a9642487069195ca, adc2e9f879454921a57a92e56e00a7e9; no voters:
I20260929 02:52:39.802136 13538 raft_consensus.cc:2781] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:39.802532 13538 raft_consensus.cc:686] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 1 LEADER]: Becoming Leader. State: Replica: adc2e9f879454921a57a92e56e00a7e9, State: Running, Role: LEADER
I20260929 02:52:39.802969 13538 consensus_queue.cc:237] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } }
I20260929 02:52:39.807461 13065 catalog_manager.cc:5495] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to adc2e9f879454921a57a92e56e00a7e9 (127.8.57.2). New cstate: current_term: 1 leader_uuid: "adc2e9f879454921a57a92e56e00a7e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adc2e9f879454921a57a92e56e00a7e9" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 41689 } health_report { overall_health: HEALTHY } } }
I20260929 02:52:39.903045 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.901109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.8.57.2:41689)
0929 02:52:39.901407 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.901411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.901428 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:39.901643 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.901646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.901724 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.901808 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.902560 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.902565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.902569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.902580 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.902606 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.902616 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.902705 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.902765 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.902808 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260929 02:52:39.910920 13207 raft_consensus.cc:1261] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 1 FOLLOWER]: Refusing update from remote peer adc2e9f879454921a57a92e56e00a7e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:39.910933 13467 raft_consensus.cc:1261] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 1 FOLLOWER]: Refusing update from remote peer adc2e9f879454921a57a92e56e00a7e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:39.911777 13541 consensus_queue.cc:1035] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84202331fa604302a9642487069195ca" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:39.912160 13538 consensus_queue.cc:1035] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbca2563b634493fbbb12ab764ad89dd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:39.935093 13066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:39.935360 13066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:39.938827 13066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec2b9251603340fabca5a82ae73dc9a4: Sending DeleteTablet for 3 replicas of tablet a33510b67f754fa78bdf65b6c02a9949
I20260929 02:52:39.940065 13317 tablet_service.cc:1514] Processing DeleteTablet for tablet a33510b67f754fa78bdf65b6c02a9949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:39 UTC) from {username='slave'} at 127.0.0.1:56510
I20260929 02:52:39.940001 13447 tablet_service.cc:1514] Processing DeleteTablet for tablet a33510b67f754fa78bdf65b6c02a9949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:39 UTC) from {username='slave'} at 127.0.0.1:48628
I20260929 02:52:39.940279 13187 tablet_service.cc:1514] Processing DeleteTablet for tablet a33510b67f754fa78bdf65b6c02a9949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:39 UTC) from {username='slave'} at 127.0.0.1:48654
I20260929 02:52:39.942130 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13125
I20260929 02:52:39.951500 13553 tablet_replica.cc:330] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: stopping tablet replica
I20260929 02:52:39.952162 13553 raft_consensus.cc:2229] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:39.953080 13553 raft_consensus.cc:2258] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:39.955271 13553 ts_tablet_manager.cc:1905] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:39.955633 13554 tablet_replica.cc:330] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: stopping tablet replica
I20260929 02:52:39.956334 13554 raft_consensus.cc:2229] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:39.956847 13554 raft_consensus.cc:2258] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:39.958323 13555 tablet_replica.cc:330] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: stopping tablet replica
I20260929 02:52:39.959090 13555 raft_consensus.cc:2229] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:39.959609 13555 raft_consensus.cc:2258] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:39.960080 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:39.943480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.8.57.1:43339)
0929 02:52:39.943666 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:39.943672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:39.957725 (+ 14053us) server_negotiation.cc:407] Connection header received
0929 02:52:39.957900 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:39.957905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:39.957996 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:39.958112 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:39.959272 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:39.959279 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:39.959284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:39.959305 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:39.959344 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:39.959359 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:39.959469 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:39.959701 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 02:52:39.959770 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:52:39.961153 13247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:39.961902 13555 ts_tablet_manager.cc:1905] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:39.965844 13553 ts_tablet_manager.cc:1918] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:39.966121 13553 log.cc:1196] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/a33510b67f754fa78bdf65b6c02a9949
I20260929 02:52:39.966753 13553 ts_tablet_manager.cc:1939] T a33510b67f754fa78bdf65b6c02a9949 P adc2e9f879454921a57a92e56e00a7e9: Deleting consensus metadata
I20260929 02:52:39.968591 13052 catalog_manager.cc:4841] TS adc2e9f879454921a57a92e56e00a7e9 (127.8.57.2:41689): tablet a33510b67f754fa78bdf65b6c02a9949 (table test-table [id=be04ba60b0a24762b9f57ca52abbc7ea]) successfully deleted
I20260929 02:52:39.968809 13555 ts_tablet_manager.cc:1918] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:39.968942 13555 log.cc:1196] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/a33510b67f754fa78bdf65b6c02a9949
I20260929 02:52:39.969376 13555 ts_tablet_manager.cc:1939] T a33510b67f754fa78bdf65b6c02a9949 P fbca2563b634493fbbb12ab764ad89dd: Deleting consensus metadata
I20260929 02:52:39.970654 13050 catalog_manager.cc:4841] TS fbca2563b634493fbbb12ab764ad89dd (127.8.57.3:36481): tablet a33510b67f754fa78bdf65b6c02a9949 (table test-table [id=be04ba60b0a24762b9f57ca52abbc7ea]) successfully deleted
I20260929 02:52:39.962100 13554 ts_tablet_manager.cc:1905] T a33510b67f754fa78bdf65b6c02a9949 P 84202331fa604302a9642487069195ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:40.094998 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13125
W20260929 02:52:40.108623 13052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.57.1:43339 (error 108)
I20260929 02:52:40.108830 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13255
I20260929 02:52:40.111804 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.110395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48664)
0929 02:52:40.111448 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.111589 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":815,"threads_started":1}
I20260929 02:52:40.112157 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.109828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.8.57.2:41689)
0929 02:52:40.110008 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:40.110013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:40.110030 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:40.110149 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:40.110155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:40.110223 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:40.110318 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:40.111486 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:40.111492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:40.111495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:40.111510 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:40.111542 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:40.111557 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:40.111676 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:40.111779 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 02:52:40.111838 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 02:52:40.112224 13052 catalog_manager.cc:4584] TS 84202331fa604302a9642487069195ca (127.8.57.1:43339): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a33510b67f754fa78bdf65b6c02a9949: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
I20260929 02:52:40.112857 13377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:40.165956 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.165482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48680)
0929 02:52:40.165698 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.165796 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 02:52:40.166919 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.166585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48690)
0929 02:52:40.166721 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.166791 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 02:52:40.220181 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13255
I20260929 02:52:40.227213 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.226772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48704)
0929 02:52:40.226969 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.227055 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 02:52:40.228236 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.227894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48712)
0929 02:52:40.228015 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.228086 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 02:52:40.234361 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13385
I20260929 02:52:40.236737 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.235055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48632 (local address 127.8.57.3:36481)
0929 02:52:40.235199 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:40.235203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:40.235220 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:40.235328 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:40.235333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:40.235404 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:40.235491 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:40.236285 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:40.236289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:40.236292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:40.236307 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:40.236338 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:40.236350 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:40.236450 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:40.236490 (+ 40us) server_negotiation.cc:299] Negotiation successful
0929 02:52:40.236533 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:52:40.237305 13507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:40.309463 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.308958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48728)
0929 02:52:40.309213 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.309302 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 02:52:40.310760 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.310262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43339 (local address 127.0.0.1:48734)
0929 02:52:40.310499 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.310603 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260929 02:52:40.322870 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13385
I20260929 02:52:40.334719 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13035
I20260929 02:52:40.335855 13096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:40.438791 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13035
2026-09-29T02:52:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2811 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 02:52:40.474687 8420 mini_oidc.cc:152] Starting JWKS server
I20260929 02:52:40.477182 8420 webserver.cc:460] Webserver started at https://127.0.0.1:46763/ using document root <none> and password file <none>
I20260929 02:52:40.477476 8420 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:52:40.478055 8420 webserver.cc:460] Webserver started at http://127.0.0.1:45741/ using document root <none> and password file <none>
2026-09-29T02:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:40Z Disabled control of system clock
I20260929 02:52:40.500933 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36297
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:34481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:36297
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46763/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/testchainca.cert with env {}
W20260929 02:52:40.805855 13568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:40.806192 13568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:40.806322 13568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:40.813484 13568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:40.813606 13568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:40.813670 13568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:40.813745 13568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:40.813803 13568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:52:40.813884 13568 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:52:40.813934 13568 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46763/jwks.json
I20260929 02:52:40.823758 13568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:36297
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46763/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:40.825052 13568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:40.826321 13568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:40.837494 13573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:40.837437 13576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:40.837435 13574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:40.838604 13568 server_base.cc:1034] running on GCE node
I20260929 02:52:40.839332 13568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:40.840868 13568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:40.842082 13568 hybrid_clock.cc:648] HybridClock initialized: now 1790650360842039 us; error 28 us; skew 500 ppm
I20260929 02:52:40.842422 13568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:40.844632 13568 webserver.cc:460] Webserver started at http://127.8.57.62:46583/ using document root <none> and password file <none>
I20260929 02:52:40.845346 13568 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:40.845443 13568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:40.845741 13568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:40.848676 13568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "87f667a51ae94074b36204d74accf3f4"
format_stamp: "Formatted at 2026-09-29 02:52:40 on dist-test-slave-1dkt"
server_key: "edc7f0e40912ff7fd49b2f4e2ba2bfb9"
server_key_iv: "5d4e18353f7fd823c05d8043b4ac00ae"
server_key_version: "encryptionkey@0"
I20260929 02:52:40.849443 13568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "87f667a51ae94074b36204d74accf3f4"
format_stamp: "Formatted at 2026-09-29 02:52:40 on dist-test-slave-1dkt"
server_key: "edc7f0e40912ff7fd49b2f4e2ba2bfb9"
server_key_iv: "5d4e18353f7fd823c05d8043b4ac00ae"
server_key_version: "encryptionkey@0"
I20260929 02:52:40.854405 13568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:40.857790 13582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:40.859110 13568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:40.859263 13568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "87f667a51ae94074b36204d74accf3f4"
format_stamp: "Formatted at 2026-09-29 02:52:40 on dist-test-slave-1dkt"
server_key: "edc7f0e40912ff7fd49b2f4e2ba2bfb9"
server_key_iv: "5d4e18353f7fd823c05d8043b4ac00ae"
server_key_version: "encryptionkey@0"
I20260929 02:52:40.859453 13568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:40.870978 13568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:40.871730 13568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:40.872020 13568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:40.888012 13568 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:36297
I20260929 02:52:40.888047 13633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:36297 every 8 connection(s)
I20260929 02:52:40.889317 13568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:40.892877 13634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:40.896211 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13568
I20260929 02:52:40.896396 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:40.896984 8420 external_mini_cluster.cc:1428] Setting key c7eddace2338d555feb1056401889593
I20260929 02:52:40.905840 13634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4: Bootstrap starting.
I20260929 02:52:40.909632 13634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:40.910925 13634 log.cc:824] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:40.911226 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.898763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42486 (local address 127.8.57.62:36297)
0929 02:52:40.899735 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:40.899749 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:40.899799 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:52:40.901488 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:40.901516 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:40.902137 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:40.902565 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:40.903096 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:40.905012 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:40.906303 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:40.907261 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:40.907473 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:40.908486 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:40.908497 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:40.908514 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:40.909202 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:40.909316 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:40.909340 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:40.909455 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:40.909588 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 02:52:40.909923 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":279,"threads_started":1}
I20260929 02:52:40.914060 13634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4: No bootstrap required, opened a new log
I20260929 02:52:40.915124 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:52:40.921555 13634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f667a51ae94074b36204d74accf3f4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36297 } }
I20260929 02:52:40.921886 13634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:40.921980 13634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f667a51ae94074b36204d74accf3f4, State: Initialized, Role: FOLLOWER
I20260929 02:52:40.922482 13634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f667a51ae94074b36204d74accf3f4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36297 } }
I20260929 02:52:40.922730 13634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:40.922863 13634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:40.923032 13634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:40.925233 13634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f667a51ae94074b36204d74accf3f4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36297 } }
I20260929 02:52:40.925686 13634 leader_election.cc:302] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f667a51ae94074b36204d74accf3f4; no voters:
I20260929 02:52:40.926334 13634 leader_election.cc:288] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:40.926435 13639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:40.927080 13639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [term 1 LEADER]: Becoming Leader. State: Replica: 87f667a51ae94074b36204d74accf3f4, State: Running, Role: LEADER
I20260929 02:52:40.927594 13639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f667a51ae94074b36204d74accf3f4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36297 } }
I20260929 02:52:40.928359 13634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:40.930496 13640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f667a51ae94074b36204d74accf3f4. Latest consensus state: current_term: 1 leader_uuid: "87f667a51ae94074b36204d74accf3f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f667a51ae94074b36204d74accf3f4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36297 } } }
I20260929 02:52:40.930935 13640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:40.933038 13641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87f667a51ae94074b36204d74accf3f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f667a51ae94074b36204d74accf3f4" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36297 } } }
I20260929 02:52:40.933266 13648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:40.933387 13641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:40.938298 13648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:40.946955 13648 catalog_manager.cc:1347] Generated new cluster ID: b67efe97decb4fb180766a3d407488d8
I20260929 02:52:40.947077 13648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:40.955350 13648 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:40.956629 13648 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:40.967325 13648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87f667a51ae94074b36204d74accf3f4: Generated new TSK 0
I20260929 02:52:40.968139 13648 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 02:52:41.001652 13664 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 02:52:41.001797 13664 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 02:52:41.002928 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:40.981720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.8.57.62:36297)
0929 02:52:40.981910 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:40.981917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:40.982360 (+ 443us) server_negotiation.cc:407] Connection header received
0929 02:52:40.982630 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:40.982635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:40.982739 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:40.982901 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:52:40.983376 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:40.984519 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:40.985886 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:40.986592 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:40.986722 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:40.987235 (+ 513us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:52:41.002277 (+ 15042us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 02:52:41.002636 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42502: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 02:52:41.003046 13637 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42502: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 02:52:41.004297 13663 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:40.981581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36297 (local address 127.0.0.1:42502)
0929 02:52:40.982260 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:40.982280 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:40.982425 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:40.982908 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:40.982911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:40.982921 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:52:40.983218 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:40.983226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:40.984656 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:40.984660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:40.985752 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:40.985763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:40.986697 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:40.986753 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:40.986888 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:40.986952 (+ 64us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:52:41.002573 (+ 15621us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 02:52:41.002837 (+ 264us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.57.62:36297: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":353,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.57.62:36297: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 02:52:41.005726 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13568
I20260929 02:52:41.007051 13629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:41.097393 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13568
2026-09-29T02:52:41Z chronyd exiting
I20260929 02:52:41.144479 8420 test_util.cc:175] -----------------------------------------------
I20260929 02:52:41.144659 8420 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650321430198-8420-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (674 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 02:52:41.150018 8420 mini_oidc.cc:152] Starting JWKS server
I20260929 02:52:41.152585 8420 webserver.cc:460] Webserver started at https://127.0.0.1:38437/ using document root <none> and password file <none>
I20260929 02:52:41.152877 8420 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:52:41.153743 8420 webserver.cc:460] Webserver started at http://127.0.0.1:43895/ using document root <none> and password file <none>
2026-09-29T02:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:41Z Disabled control of system clock
I20260929 02:52:41.177083 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:43889
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:43889
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38437/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/testchainca.cert with env {}
W20260929 02:52:41.493700 13671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:41.494017 13671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:41.494156 13671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:41.501353 13671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:41.501466 13671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:41.501533 13671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:41.501609 13671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:41.501669 13671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:52:41.501751 13671 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:52:41.501803 13671 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38437/jwks.json
I20260929 02:52:41.511845 13671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:43889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:43889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38437/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:41.513176 13671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:41.514456 13671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:41.525645 13676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:41.525641 13677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:41.525872 13679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:41.526369 13671 server_base.cc:1034] running on GCE node
I20260929 02:52:41.527118 13671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:41.528599 13671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:41.529800 13671 hybrid_clock.cc:648] HybridClock initialized: now 1790650361529754 us; error 43 us; skew 500 ppm
I20260929 02:52:41.530110 13671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:41.532248 13671 webserver.cc:460] Webserver started at http://127.8.57.62:39247/ using document root <none> and password file <none>
I20260929 02:52:41.532881 13671 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:41.532965 13671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:41.533234 13671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:41.535971 13671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "56a2054695a24f7d925bd1458d52af19"
format_stamp: "Formatted at 2026-09-29 02:52:41 on dist-test-slave-1dkt"
server_key: "668fb33e9e1ac4f36c88e11f9db10e52"
server_key_iv: "e35a3a2bc05c751756723f6a7a44f3cb"
server_key_version: "encryptionkey@0"
I20260929 02:52:41.536711 13671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "56a2054695a24f7d925bd1458d52af19"
format_stamp: "Formatted at 2026-09-29 02:52:41 on dist-test-slave-1dkt"
server_key: "668fb33e9e1ac4f36c88e11f9db10e52"
server_key_iv: "e35a3a2bc05c751756723f6a7a44f3cb"
server_key_version: "encryptionkey@0"
I20260929 02:52:41.541661 13671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:52:41.545149 13685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:41.546473 13671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:41.546613 13671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "56a2054695a24f7d925bd1458d52af19"
format_stamp: "Formatted at 2026-09-29 02:52:41 on dist-test-slave-1dkt"
server_key: "668fb33e9e1ac4f36c88e11f9db10e52"
server_key_iv: "e35a3a2bc05c751756723f6a7a44f3cb"
server_key_version: "encryptionkey@0"
I20260929 02:52:41.546785 13671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:41.572868 13671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:41.573715 13671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:41.573978 13671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:41.590548 13671 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:43889
I20260929 02:52:41.590572 13736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:43889 every 8 connection(s)
I20260929 02:52:41.591825 13671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:41.592921 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13671
I20260929 02:52:41.593087 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:41.593715 8420 external_mini_cluster.cc:1428] Setting key 4ca59914b430eed946a2cb35b79b2478
I20260929 02:52:41.596113 13737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:41.608891 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:41.595505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.8.57.62:43889)
0929 02:52:41.596558 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:41.596572 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:41.596722 (+ 150us) server_negotiation.cc:407] Connection header received
0929 02:52:41.598883 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:41.598912 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:41.599540 (+ 628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:41.600021 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:41.600554 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:41.602524 (+ 1970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:41.603730 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:41.604748 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:41.604911 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:41.605975 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:41.605985 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:41.606003 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:41.606732 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:41.606836 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:41.606866 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:41.606979 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:41.607230 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 02:52:41.607586 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":349,"threads_started":1}
I20260929 02:52:41.609988 13737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19: Bootstrap starting.
I20260929 02:52:41.612967 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:52:41.613720 13737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:41.614920 13737 log.cc:824] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:41.617942 13737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19: No bootstrap required, opened a new log
I20260929 02:52:41.624962 13737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a2054695a24f7d925bd1458d52af19" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43889 } }
I20260929 02:52:41.625305 13737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:41.625377 13737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a2054695a24f7d925bd1458d52af19, State: Initialized, Role: FOLLOWER
I20260929 02:52:41.625890 13737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a2054695a24f7d925bd1458d52af19" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43889 } }
I20260929 02:52:41.626147 13737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:41.626243 13737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:41.626361 13737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:41.628473 13737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a2054695a24f7d925bd1458d52af19" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43889 } }
I20260929 02:52:41.629036 13737 leader_election.cc:302] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a2054695a24f7d925bd1458d52af19; no voters:
I20260929 02:52:41.629616 13737 leader_election.cc:288] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:41.629726 13742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:41.630368 13742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [term 1 LEADER]: Becoming Leader. State: Replica: 56a2054695a24f7d925bd1458d52af19, State: Running, Role: LEADER
I20260929 02:52:41.630888 13742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a2054695a24f7d925bd1458d52af19" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43889 } }
I20260929 02:52:41.631568 13737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:41.633544 13743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a2054695a24f7d925bd1458d52af19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a2054695a24f7d925bd1458d52af19" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43889 } } }
I20260929 02:52:41.634860 13743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:41.633863 13744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a2054695a24f7d925bd1458d52af19. Latest consensus state: current_term: 1 leader_uuid: "56a2054695a24f7d925bd1458d52af19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a2054695a24f7d925bd1458d52af19" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43889 } } }
I20260929 02:52:41.635982 13744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:41.636735 13751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:41.641507 13751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:41.650128 13751 catalog_manager.cc:1347] Generated new cluster ID: cc566b0db9cb4021894a967543b6a23e
I20260929 02:52:41.650239 13751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:41.668269 13751 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:41.669593 13751 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:41.685279 13751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a2054695a24f7d925bd1458d52af19: Generated new TSK 0
I20260929 02:52:41.686411 13751 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 02:52:41.727371 13767 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 02:52:41.727484 13767 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260929 02:52:41.728869 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:41.708333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:43889 (local address 127.0.0.1:60272)
0929 02:52:41.708971 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:41.708986 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:41.709113 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:41.709592 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:41.709595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:41.709605 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:52:41.709849 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:41.709857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:41.711225 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:41.711229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:41.712355 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:41.712367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:41.713324 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:41.713383 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:41.713521 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:41.713555 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:52:41.728391 (+ 14836us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 02:52:41.728591 (+ 200us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.57.62:43889: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":355,"threads_started":1}
I20260929 02:52:41.728859 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:41.708477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.8.57.62:43889)
0929 02:52:41.708678 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:41.708684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:41.709068 (+ 384us) server_negotiation.cc:407] Connection header received
0929 02:52:41.709301 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:41.709307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:41.709403 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:41.709583 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:52:41.709985 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:41.711101 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:41.712494 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:41.713217 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:41.713342 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:41.713816 (+ 474us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:52:41.728152 (+ 14336us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 02:52:41.728542 (+ 390us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 02:52:41.728977 13740 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260929 02:52:41.730288 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13671
I20260929 02:52:41.731470 13732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:41.823921 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13671
2026-09-29T02:52:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (726 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 02:52:41.874646 8420 mini_oidc.cc:152] Starting JWKS server
I20260929 02:52:41.876469 8420 webserver.cc:460] Webserver started at https://127.0.0.1:45679/ using document root <none> and password file <none>
I20260929 02:52:41.876691 8420 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:52:41.877260 8420 webserver.cc:460] Webserver started at http://127.0.0.1:38025/ using document root <none> and password file <none>
2026-09-29T02:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:41Z Disabled control of system clock
I20260929 02:52:41.900926 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41263
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41263
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45679/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 02:52:42.208122 13774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:42.208415 13774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:42.208523 13774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:42.215777 13774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:42.215912 13774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:42.215969 13774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:42.216060 13774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:42.216163 13774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:52:42.216256 13774 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:52:42.216315 13774 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45679/jwks.json
I20260929 02:52:42.225633 13774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45679/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:42.226958 13774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:42.228792 13774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:42.240028 13780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:42.240168 13782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:42.240175 13779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:42.241079 13774 server_base.cc:1034] running on GCE node
I20260929 02:52:42.241837 13774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:42.243252 13774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:42.244444 13774 hybrid_clock.cc:648] HybridClock initialized: now 1790650362244395 us; error 43 us; skew 500 ppm
I20260929 02:52:42.244773 13774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:42.247011 13774 webserver.cc:460] Webserver started at http://127.8.57.62:34195/ using document root <none> and password file <none>
I20260929 02:52:42.247671 13774 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:42.247769 13774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:42.248093 13774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:42.250941 13774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "f351cf43c7a74f3fbedf9699f4561f44"
format_stamp: "Formatted at 2026-09-29 02:52:42 on dist-test-slave-1dkt"
server_key: "21b2d88345f53162df114ac67f634c15"
server_key_iv: "5a3459e4487276362bee791ebb4cba88"
server_key_version: "encryptionkey@0"
I20260929 02:52:42.251701 13774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "f351cf43c7a74f3fbedf9699f4561f44"
format_stamp: "Formatted at 2026-09-29 02:52:42 on dist-test-slave-1dkt"
server_key: "21b2d88345f53162df114ac67f634c15"
server_key_iv: "5a3459e4487276362bee791ebb4cba88"
server_key_version: "encryptionkey@0"
I20260929 02:52:42.256609 13774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:52:42.259799 13788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:42.261243 13774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:52:42.261397 13774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "f351cf43c7a74f3fbedf9699f4561f44"
format_stamp: "Formatted at 2026-09-29 02:52:42 on dist-test-slave-1dkt"
server_key: "21b2d88345f53162df114ac67f634c15"
server_key_iv: "5a3459e4487276362bee791ebb4cba88"
server_key_version: "encryptionkey@0"
I20260929 02:52:42.261602 13774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:42.279263 13774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:42.280114 13774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:42.280364 13774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:42.296674 13774 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41263
I20260929 02:52:42.296680 13839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41263 every 8 connection(s)
I20260929 02:52:42.297935 13774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:42.301543 13840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:42.306490 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13774
I20260929 02:52:42.306718 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:42.307453 8420 external_mini_cluster.cc:1428] Setting key 0b98f2a96fdf1b48f53b60ec5549663f
I20260929 02:52:42.312649 13840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44: Bootstrap starting.
I20260929 02:52:42.315572 13840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:42.316588 13840 log.cc:824] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:42.319216 13840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44: No bootstrap required, opened a new log
I20260929 02:52:42.319998 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:42.309347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.8.57.62:41263)
0929 02:52:42.310369 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:42.310380 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:42.310424 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:52:42.312119 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:42.312142 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:42.312585 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:42.312953 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:42.313496 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:42.314731 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:42.315972 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:42.316632 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:42.316785 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:42.317840 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:42.317846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:42.317860 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:42.318356 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:42.318430 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:42.318449 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:42.318547 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:42.318763 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 02:52:42.319040 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":365,"threads_started":1}
I20260929 02:52:42.323697 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:52:42.327582 13840 raft_consensus.cc:348] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f351cf43c7a74f3fbedf9699f4561f44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41263 } }
I20260929 02:52:42.327937 13840 raft_consensus.cc:374] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:42.328034 13840 raft_consensus.cc:729] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f351cf43c7a74f3fbedf9699f4561f44, State: Initialized, Role: FOLLOWER
I20260929 02:52:42.328545 13840 consensus_queue.cc:260] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f351cf43c7a74f3fbedf9699f4561f44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41263 } }
I20260929 02:52:42.328751 13840 raft_consensus.cc:388] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:42.328867 13840 raft_consensus.cc:482] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:42.329012 13840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:42.331104 13840 raft_consensus.cc:504] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f351cf43c7a74f3fbedf9699f4561f44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41263 } }
I20260929 02:52:42.331560 13840 leader_election.cc:302] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f351cf43c7a74f3fbedf9699f4561f44; no voters:
I20260929 02:52:42.332131 13840 leader_election.cc:288] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:42.332258 13845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:42.333228 13845 raft_consensus.cc:686] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [term 1 LEADER]: Becoming Leader. State: Replica: f351cf43c7a74f3fbedf9699f4561f44, State: Running, Role: LEADER
I20260929 02:52:42.333709 13845 consensus_queue.cc:237] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f351cf43c7a74f3fbedf9699f4561f44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41263 } }
I20260929 02:52:42.334111 13840 sys_catalog.cc:564] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:42.336359 13847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f351cf43c7a74f3fbedf9699f4561f44. Latest consensus state: current_term: 1 leader_uuid: "f351cf43c7a74f3fbedf9699f4561f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f351cf43c7a74f3fbedf9699f4561f44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41263 } } }
I20260929 02:52:42.336699 13847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:42.337154 13846 sys_catalog.cc:455] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f351cf43c7a74f3fbedf9699f4561f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f351cf43c7a74f3fbedf9699f4561f44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41263 } } }
I20260929 02:52:42.337453 13846 sys_catalog.cc:458] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:42.339079 13853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:42.344278 13853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:42.352360 13853 catalog_manager.cc:1347] Generated new cluster ID: 4cf545c3c1d747479b0a9f1ef93e03dc
I20260929 02:52:42.352473 13853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:42.359063 13853 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:42.360862 13853 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:42.379046 13853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f351cf43c7a74f3fbedf9699f4561f44: Generated new TSK 0
I20260929 02:52:42.380002 13853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:52:42.406517 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:42.398476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.8.57.62:41263)
0929 02:52:42.398678 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:42.398684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:42.398970 (+ 286us) server_negotiation.cc:407] Connection header received
0929 02:52:42.399165 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:42.399170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:42.399255 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:42.399411 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:52:42.399854 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:42.400998 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:42.402295 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:42.402995 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:42.403113 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:42.403566 (+ 453us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:52:42.405871 (+ 2305us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 02:52:42.406214 (+ 343us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":83}
W20260929 02:52:42.406661 13843 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 02:52:42.408190 13869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:42.398319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41263 (local address 127.0.0.1:60956)
0929 02:52:42.398878 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:42.398894 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:42.399005 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:42.399409 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:42.399412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:42.399420 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:52:42.399663 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:42.399671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:42.401129 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:42.401133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:42.402161 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:42.402172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:42.403109 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:42.403168 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:42.403289 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:42.403320 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:52:42.406103 (+ 2783us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 02:52:42.407747 (+ 1644us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.57.62:41263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":1360,"thread_start_us":294,"threads_started":1}
I20260929 02:52:42.409356 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13774
I20260929 02:52:42.410684 13835 generic_service.cc:196] Checking for leaks (request via RPC)
E20260929 02:52:42.420897 13870 webserver.cc:533] Webserver: sslize failed:
E20260929 02:52:42.421016 13870 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 02:52:42.502754 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13774
2026-09-29T02:52:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (678 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T02:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:42Z Disabled control of system clock
I20260929 02:52:42.576121 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42739
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42739
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/testchainca.cert with env {}
W20260929 02:52:42.884354 13875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:42.884645 13875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:42.884738 13875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:42.891863 13875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:42.891992 13875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:42.892055 13875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:42.892112 13875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:42.892153 13875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:52:42.892211 13875 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:52:42.892248 13875 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 02:52:42.902290 13875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:42.903571 13875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:42.904826 13875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:42.916141 13880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:42.916102 13883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:42.916193 13881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:42.916914 13875 server_base.cc:1034] running on GCE node
I20260929 02:52:42.917618 13875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:42.918931 13875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:42.920122 13875 hybrid_clock.cc:648] HybridClock initialized: now 1790650362920082 us; error 31 us; skew 500 ppm
I20260929 02:52:42.920490 13875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:52:42.922653 13875 webserver.cc:460] Webserver started at http://127.8.57.62:41667/ using document root <none> and password file <none>
I20260929 02:52:42.923309 13875 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:42.923405 13875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:42.923696 13875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:42.926504 13875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "e95b7830b5854eba8fa0a47bf19606a9"
format_stamp: "Formatted at 2026-09-29 02:52:42 on dist-test-slave-1dkt"
server_key: "4ce19d5bae6f9d2fdd634faca37f50fc"
server_key_iv: "bf3b9d094edc709bcd33488a5c6bc66c"
server_key_version: "encryptionkey@0"
I20260929 02:52:42.927258 13875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "e95b7830b5854eba8fa0a47bf19606a9"
format_stamp: "Formatted at 2026-09-29 02:52:42 on dist-test-slave-1dkt"
server_key: "4ce19d5bae6f9d2fdd634faca37f50fc"
server_key_iv: "bf3b9d094edc709bcd33488a5c6bc66c"
server_key_version: "encryptionkey@0"
I20260929 02:52:42.932101 13875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:52:42.935271 13889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:42.936579 13875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:42.936733 13875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "e95b7830b5854eba8fa0a47bf19606a9"
format_stamp: "Formatted at 2026-09-29 02:52:42 on dist-test-slave-1dkt"
server_key: "4ce19d5bae6f9d2fdd634faca37f50fc"
server_key_iv: "bf3b9d094edc709bcd33488a5c6bc66c"
server_key_version: "encryptionkey@0"
I20260929 02:52:42.936923 13875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:42.966998 13875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:42.967898 13875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:42.968179 13875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:42.984382 13875 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42739
I20260929 02:52:42.984400 13940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42739 every 8 connection(s)
I20260929 02:52:42.985879 13875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:42.989555 13941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:42.992549 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13875
I20260929 02:52:42.992745 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:42.993535 8420 external_mini_cluster.cc:1428] Setting key 66cbb7718445b705f749658689557ad6
I20260929 02:52:43.001778 13941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9: Bootstrap starting.
I20260929 02:52:43.005241 13941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:43.006383 13941 log.cc:824] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:43.006757 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:42.995447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.8.57.62:42739)
0929 02:52:42.996421 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:42.996431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:42.996472 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:52:42.998053 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:42.998076 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:42.998558 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:42.998882 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:42.999435 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:43.000739 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:43.002478 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:43.003169 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:43.003328 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:43.004502 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:43.004509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:43.004526 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:52:43.005024 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:43.005127 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:52:43.005150 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:43.005235 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:43.005471 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 02:52:43.005794 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":353,"threads_started":1}
I20260929 02:52:43.009544 13941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9: No bootstrap required, opened a new log
I20260929 02:52:43.010854 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:52:43.018060 13941 raft_consensus.cc:348] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95b7830b5854eba8fa0a47bf19606a9" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42739 } }
I20260929 02:52:43.018417 13941 raft_consensus.cc:374] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:43.018491 13941 raft_consensus.cc:729] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e95b7830b5854eba8fa0a47bf19606a9, State: Initialized, Role: FOLLOWER
I20260929 02:52:43.018998 13941 consensus_queue.cc:260] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95b7830b5854eba8fa0a47bf19606a9" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42739 } }
I20260929 02:52:43.019197 13941 raft_consensus.cc:388] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:43.019299 13941 raft_consensus.cc:482] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:43.019445 13941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:43.021780 13941 raft_consensus.cc:504] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95b7830b5854eba8fa0a47bf19606a9" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42739 } }
I20260929 02:52:43.022257 13941 leader_election.cc:302] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e95b7830b5854eba8fa0a47bf19606a9; no voters:
I20260929 02:52:43.022823 13941 leader_election.cc:288] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:43.022948 13946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:43.023612 13946 raft_consensus.cc:686] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [term 1 LEADER]: Becoming Leader. State: Replica: e95b7830b5854eba8fa0a47bf19606a9, State: Running, Role: LEADER
I20260929 02:52:43.024268 13946 consensus_queue.cc:237] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95b7830b5854eba8fa0a47bf19606a9" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42739 } }
I20260929 02:52:43.025180 13941 sys_catalog.cc:564] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:43.027242 13948 sys_catalog.cc:455] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e95b7830b5854eba8fa0a47bf19606a9. Latest consensus state: current_term: 1 leader_uuid: "e95b7830b5854eba8fa0a47bf19606a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95b7830b5854eba8fa0a47bf19606a9" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42739 } } }
I20260929 02:52:43.028789 13948 sys_catalog.cc:458] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:43.027612 13947 sys_catalog.cc:455] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e95b7830b5854eba8fa0a47bf19606a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95b7830b5854eba8fa0a47bf19606a9" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42739 } } }
I20260929 02:52:43.028909 13947 sys_catalog.cc:458] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:43.031145 13954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:43.035977 13954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:43.043702 13954 catalog_manager.cc:1347] Generated new cluster ID: e4fe9961308140a5b579d13bca999fc8
I20260929 02:52:43.043809 13954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:43.058379 13954 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:43.059509 13954 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:43.069453 13954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e95b7830b5854eba8fa0a47bf19606a9: Generated new TSK 0
I20260929 02:52:43.070192 13954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:52:43.552698 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:43.089802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.8.57.62:42739)
0929 02:52:43.089978 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:43.089984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:43.090437 (+ 453us) server_negotiation.cc:407] Connection header received
0929 02:52:43.090625 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:43.090631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:43.090716 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:43.090877 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:52:43.091254 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:43.092420 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:43.093751 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:43.094433 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:43.094566 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:43.094989 (+ 423us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:52:43.552143 (+457154us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 02:52:43.552455 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52264: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 02:52:43.552793 13944 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52264: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 02:52:43.552986 13970 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:43.089751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42739 (local address 127.0.0.1:52264)
0929 02:52:43.090350 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:43.090364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:43.090474 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:43.090882 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:43.090885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:43.090893 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:52:43.091127 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:43.091135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:43.092542 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:43.092546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:43.093624 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:43.093634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:43.094556 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:43.094599 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:43.094727 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:43.094759 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:52:43.552403 (+457644us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 02:52:43.552649 (+ 246us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.57.62:42739: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":326,"threads_started":1}
I20260929 02:52:43.554486 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13875
I20260929 02:52:43.555444 13936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:43.655560 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13875
2026-09-29T02:52:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1134 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (384 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (389 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:44 dist-test-slave-1dkt krb5kdc[13985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:44 dist-test-slave-1dkt krb5kdc[13985](info): set up 2 sockets
Sep 29 02:52:44 dist-test-slave-1dkt krb5kdc[13985](info): commencing operation
krb5kdc: starting...
W20260929 02:52:45.429176 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:45 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:52:45.610885 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:46789
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:46789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:45.919752 13998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:45.920060 13998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:45.920192 13998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:45.927187 13998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:45.927286 13998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:45.927356 13998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:45.927428 13998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:45.927486 13998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:45.937196 13998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:46789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:46789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:45.938438 13998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:45.939711 13998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:45.950366 14003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:45.950407 14004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:45.950433 14006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:45.951174 13998 server_base.cc:1034] running on GCE node
I20260929 02:52:45.951918 13998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:45.953408 13998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:45.954658 13998 hybrid_clock.cc:648] HybridClock initialized: now 1790650365954596 us; error 52 us; skew 500 ppm
I20260929 02:52:45.955018 13998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:45 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650365, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:45.957919 13998 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:52:45.959271 13998 webserver.cc:460] Webserver started at http://127.8.57.62:46683/ using document root <none> and password file <none>
I20260929 02:52:45.959950 13998 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:45.960067 13998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:45.960343 13998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:45.963142 13998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "b6b6cefae4084b8d8ae0ed7409d30b05"
format_stamp: "Formatted at 2026-09-29 02:52:45 on dist-test-slave-1dkt"
server_key: "be9d02bb6b21cd5479161e0624b96509"
server_key_iv: "7bcc903f87b67e19e2f6fa836e8964d0"
server_key_version: "encryptionkey@0"
I20260929 02:52:45.963852 13998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "b6b6cefae4084b8d8ae0ed7409d30b05"
format_stamp: "Formatted at 2026-09-29 02:52:45 on dist-test-slave-1dkt"
server_key: "be9d02bb6b21cd5479161e0624b96509"
server_key_iv: "7bcc903f87b67e19e2f6fa836e8964d0"
server_key_version: "encryptionkey@0"
I20260929 02:52:45.968816 13998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260929 02:52:45.971982 14013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:45.973191 13998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:52:45.973344 13998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "b6b6cefae4084b8d8ae0ed7409d30b05"
format_stamp: "Formatted at 2026-09-29 02:52:45 on dist-test-slave-1dkt"
server_key: "be9d02bb6b21cd5479161e0624b96509"
server_key_iv: "7bcc903f87b67e19e2f6fa836e8964d0"
server_key_version: "encryptionkey@0"
I20260929 02:52:45.973512 13998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:45.987427 13998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:45.991094 13998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:45.991343 13998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:46.007496 13998 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:46789
I20260929 02:52:46.007543 14064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:46789 every 8 connection(s)
I20260929 02:52:46.008859 13998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:46.012580 14065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:46.017802 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 13998
I20260929 02:52:46.017999 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:46.018744 8420 external_mini_cluster.cc:1428] Setting key 94b72891410be77e533c342c0e934f23
I20260929 02:52:46.024224 14065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05: Bootstrap starting.
I20260929 02:52:46.028024 14065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:46.029258 14065 log.cc:824] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:46.032330 14065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05: No bootstrap required, opened a new log
Sep 29 02:52:46 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:46.041049 14065 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46789 } }
I20260929 02:52:46.041414 14065 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:46.041527 14065 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b6cefae4084b8d8ae0ed7409d30b05, State: Initialized, Role: FOLLOWER
I20260929 02:52:46.042243 14065 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46789 } }
I20260929 02:52:46.042471 14065 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:46.042601 14065 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:46.042750 14065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:46.043535 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:46.021152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.8.57.62:46789)
0929 02:52:46.022136 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:46.022153 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:46.022529 (+ 376us) server_negotiation.cc:407] Connection header received
0929 02:52:46.024927 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:46.024955 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:46.025440 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:46.025867 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:46.027113 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:46.029052 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:46.030662 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:46.031640 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:46.031818 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:46.036100 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:46.036108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:46.036125 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:46.036174 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:46.039270 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:46.040127 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:46.040135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:46.040142 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:46.040323 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:46.040710 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:46.040713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:46.040715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:46.041257 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:46.041509 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:46.041968 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 02:52:46.042377 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":453,"threads_started":1}
I20260929 02:52:46.045322 14065 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46789 } }
I20260929 02:52:46.045881 14065 leader_election.cc:302] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b6cefae4084b8d8ae0ed7409d30b05; no voters:
I20260929 02:52:46.046559 14065 leader_election.cc:288] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:46.046697 14070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:46.047346 14070 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [term 1 LEADER]: Becoming Leader. State: Replica: b6b6cefae4084b8d8ae0ed7409d30b05, State: Running, Role: LEADER
I20260929 02:52:46.047849 14070 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46789 } }
I20260929 02:52:46.048615 14065 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:46.051076 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46789 } } }
I20260929 02:52:46.051451 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:46.056099 14072 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b6cefae4084b8d8ae0ed7409d30b05. Latest consensus state: current_term: 1 leader_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6cefae4084b8d8ae0ed7409d30b05" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46789 } } }
I20260929 02:52:46.056361 14072 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05 [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:46.056897 14078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:46.061488 14078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:46.069854 14078 catalog_manager.cc:1347] Generated new cluster ID: 069f4b63cf64415d836cba84c347f29b
I20260929 02:52:46.069962 14078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:46.105832 14078 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:46.107149 14078 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:46.121750 14078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6b6cefae4084b8d8ae0ed7409d30b05: Generated new TSK 0
I20260929 02:52:46.122545 14078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:52:46.177706 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46789
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:46.489981 14091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:46.490262 14091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:46.490403 14091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:46.497701 14091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:46.497825 14091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:46.497983 14091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:46.507532 14091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:46.508832 14091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:46.510097 14091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:46.522234 14099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:46.522289 14097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:46.522478 14096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:46.523202 14091 server_base.cc:1034] running on GCE node
I20260929 02:52:46.523769 14091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:46.524681 14091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:46.525883 14091 hybrid_clock.cc:648] HybridClock initialized: now 1790650366525835 us; error 36 us; skew 500 ppm
I20260929 02:52:46.526207 14091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:46 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:46.529122 14091 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:52:46.530462 14091 webserver.cc:460] Webserver started at http://127.8.57.1:38161/ using document root <none> and password file <none>
I20260929 02:52:46.531090 14091 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:46.531198 14091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:46.531464 14091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:46.534200 14091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "320b253eda6d4c6dba02bd7b7f0fd0e8"
format_stamp: "Formatted at 2026-09-29 02:52:46 on dist-test-slave-1dkt"
server_key: "9eeddcd5dbd6169a1066844e8e01b3e1"
server_key_iv: "3a9e0c11f9677e75d25e2beafeeb4ae1"
server_key_version: "encryptionkey@0"
I20260929 02:52:46.534893 14091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "320b253eda6d4c6dba02bd7b7f0fd0e8"
format_stamp: "Formatted at 2026-09-29 02:52:46 on dist-test-slave-1dkt"
server_key: "9eeddcd5dbd6169a1066844e8e01b3e1"
server_key_iv: "3a9e0c11f9677e75d25e2beafeeb4ae1"
server_key_version: "encryptionkey@0"
I20260929 02:52:46.539535 14091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 02:52:46.542762 14106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:46.544140 14091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:46.544281 14091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "320b253eda6d4c6dba02bd7b7f0fd0e8"
format_stamp: "Formatted at 2026-09-29 02:52:46 on dist-test-slave-1dkt"
server_key: "9eeddcd5dbd6169a1066844e8e01b3e1"
server_key_iv: "3a9e0c11f9677e75d25e2beafeeb4ae1"
server_key_version: "encryptionkey@0"
I20260929 02:52:46.544459 14091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:46.561882 14091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:46.565665 14091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:46.565950 14091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:46.566915 14091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:46.568233 14091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:46.568310 14091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:46.568428 14091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:46.568480 14091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:46.597448 14091 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:35677
I20260929 02:52:46.597492 14218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:35677 every 8 connection(s)
I20260929 02:52:46.598690 14091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:46.607121 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14091
I20260929 02:52:46.607290 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:46.607887 8420 external_mini_cluster.cc:1428] Setting key b4c7f6fff1fc3cb03a4cae64a42b99cb
Sep 29 02:52:46 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:46.617413 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:46.601019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:41843 (local address 127.8.57.62:46789)
0929 02:52:46.601421 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:46.601425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:46.602268 (+ 843us) server_negotiation.cc:407] Connection header received
0929 02:52:46.603999 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:46.604006 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:46.604092 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:46.604185 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:46.605592 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:46.606341 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:46.607909 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:46.608389 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:46.608515 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:46.612150 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:46.612156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:46.612159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:46.612191 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:46.614528 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:46.615215 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:46.615221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:46.615223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:46.615337 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:46.615732 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:46.615737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:46.615740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:46.616040 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:46.616181 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:46.616831 (+ 650us) server_negotiation.cc:299] Negotiation successful
0929 02:52:46.617050 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":214,"threads_started":1}
I20260929 02:52:46.618360 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:46.601435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46789 (local address 127.8.57.1:41843)
0929 02:52:46.602119 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:46.602153 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:46.603726 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:46.604413 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:46.604421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:46.604906 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:46.605433 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:46.605449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:46.606487 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:46.606490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:46.607760 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:46.607772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:46.608604 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:46.608609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:46.608888 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:46.609727 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:46.609764 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:46.611919 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:46.614666 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:46.614676 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:46.614693 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:46.615098 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:46.615437 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:46.615442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:46.615446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:46.615631 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:46.616156 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:46.616164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:46.616490 (+ 326us) client_negotiation.cc:769] Sending connection context
0929 02:52:46.616740 (+ 250us) client_negotiation.cc:240] Negotiation successful
0929 02:52:46.617248 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":271,"threads_started":1}
I20260929 02:52:46.619961 14219 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46789
I20260929 02:52:46.620285 14219 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:46.621205 14219 heartbeater.cc:511] Master 127.8.57.62:46789 requested a full tablet report, sending...
I20260929 02:52:46.623780 14030 ts_manager.cc:194] Registered new tserver with Master: 320b253eda6d4c6dba02bd7b7f0fd0e8 (127.8.57.1:35677)
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:52:46.625838 14030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:41843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:52:46.649436 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46789
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:46.955412 14225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:46.955667 14225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:46.955802 14225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:46.962860 14225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:46.962989 14225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:46.963157 14225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:46.972824 14225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:46.974031 14225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:46.975394 14225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:46.987356 14230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:46.987375 14231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:46.987437 14233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:46.987937 14225 server_base.cc:1034] running on GCE node
I20260929 02:52:46.988519 14225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:46.989391 14225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:46.990587 14225 hybrid_clock.cc:648] HybridClock initialized: now 1790650366990536 us; error 45 us; skew 500 ppm
I20260929 02:52:46.990875 14225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:46 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:46.993705 14225 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:52:46.995083 14225 webserver.cc:460] Webserver started at http://127.8.57.2:35701/ using document root <none> and password file <none>
I20260929 02:52:46.995777 14225 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:46.995895 14225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:46.996167 14225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:46.998869 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "2fa10c5edff943e7b6f50f5c24e7b5fc"
format_stamp: "Formatted at 2026-09-29 02:52:46 on dist-test-slave-1dkt"
server_key: "a7f29b4096f5fc085f9c06fe96c7c5e5"
server_key_iv: "6c0df10d1b2ff3a9b6ddf68ebdaf593d"
server_key_version: "encryptionkey@0"
I20260929 02:52:46.999564 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "2fa10c5edff943e7b6f50f5c24e7b5fc"
format_stamp: "Formatted at 2026-09-29 02:52:46 on dist-test-slave-1dkt"
server_key: "a7f29b4096f5fc085f9c06fe96c7c5e5"
server_key_iv: "6c0df10d1b2ff3a9b6ddf68ebdaf593d"
server_key_version: "encryptionkey@0"
I20260929 02:52:47.004369 14225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 02:52:47.007856 14240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:47.009418 14225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:47.009569 14225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "2fa10c5edff943e7b6f50f5c24e7b5fc"
format_stamp: "Formatted at 2026-09-29 02:52:46 on dist-test-slave-1dkt"
server_key: "a7f29b4096f5fc085f9c06fe96c7c5e5"
server_key_iv: "6c0df10d1b2ff3a9b6ddf68ebdaf593d"
server_key_version: "encryptionkey@0"
I20260929 02:52:47.009785 14225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:47.027773 14225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:47.031370 14225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:47.031612 14225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:47.032518 14225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:47.033757 14225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:47.033844 14225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:47.033959 14225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:47.034018 14225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:47.062558 14225 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:35955
I20260929 02:52:47.062601 14352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:35955 every 8 connection(s)
I20260929 02:52:47.063791 14225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:47.066623 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14225
I20260929 02:52:47.066766 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:47.067162 8420 external_mini_cluster.cc:1428] Setting key 8dd8b16abcdfd62275b62cd4bcedefcf
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:47.083724 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.066225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:33723 (local address 127.8.57.62:46789)
0929 02:52:47.066422 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.066428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.067711 (+ 1283us) server_negotiation.cc:407] Connection header received
0929 02:52:47.069496 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.069503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.069581 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.069686 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.071210 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.072189 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.073639 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.074225 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.074349 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.078264 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.078269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.078272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.078308 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.080677 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.081502 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.081506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.081509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.081632 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.082079 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.082082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.082084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.082364 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.082491 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.083304 (+ 813us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.083480 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:52:47.085219 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.066738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46789 (local address 127.8.57.2:33723)
0929 02:52:47.067534 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:47.067589 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:47.069226 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:47.069898 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:47.069910 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:47.070451 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:47.071045 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:47.071064 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.072310 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.072313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:47.073508 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:47.073519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.074440 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.074447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:47.074729 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.075433 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:47.075462 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:47.078048 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:47.080815 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:47.080824 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:47.080841 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:47.081408 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:47.081725 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:47.081729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:47.081733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:47.081972 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:47.082490 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:47.082497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:47.082931 (+ 434us) client_negotiation.cc:769] Sending connection context
0929 02:52:47.083254 (+ 323us) client_negotiation.cc:240] Negotiation successful
0929 02:52:47.083807 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:47.086619 14353 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46789
I20260929 02:52:47.086891 14353 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:47.087580 14353 heartbeater.cc:511] Master 127.8.57.62:46789 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
I20260929 02:52:47.089011 14030 ts_manager.cc:194] Registered new tserver with Master: 2fa10c5edff943e7b6f50f5c24e7b5fc (127.8.57.2:35955)
I20260929 02:52:47.089809 14030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:33723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:52:47.108314 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46789
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:47.421372 14358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:47.421648 14358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:47.421738 14358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:47.428777 14358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:47.428886 14358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:47.429003 14358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:47.438484 14358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:47.439690 14358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:47.440941 14358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:47.453411 14363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:47.453449 14366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:47.453404 14364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:47.454159 14358 server_base.cc:1034] running on GCE node
I20260929 02:52:47.454691 14358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:47.455593 14358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:47.456785 14358 hybrid_clock.cc:648] HybridClock initialized: now 1790650367456731 us; error 49 us; skew 500 ppm
I20260929 02:52:47.457098 14358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.459980 14358 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:52:47.461337 14358 webserver.cc:460] Webserver started at http://127.8.57.3:35831/ using document root <none> and password file <none>
I20260929 02:52:47.461944 14358 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:47.462061 14358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:47.462325 14358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:47.465114 14358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "bc7731a49d3447d8a5442fb81458e835"
format_stamp: "Formatted at 2026-09-29 02:52:47 on dist-test-slave-1dkt"
server_key: "d3f6a3eeb78900f71ec427a7493d12cf"
server_key_iv: "b5fa2a51b1b85085583f95fb696acf84"
server_key_version: "encryptionkey@0"
I20260929 02:52:47.465821 14358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "bc7731a49d3447d8a5442fb81458e835"
format_stamp: "Formatted at 2026-09-29 02:52:47 on dist-test-slave-1dkt"
server_key: "d3f6a3eeb78900f71ec427a7493d12cf"
server_key_iv: "b5fa2a51b1b85085583f95fb696acf84"
server_key_version: "encryptionkey@0"
I20260929 02:52:47.470649 14358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:52:47.473860 14373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:47.475096 14358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:47.475237 14358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "bc7731a49d3447d8a5442fb81458e835"
format_stamp: "Formatted at 2026-09-29 02:52:47 on dist-test-slave-1dkt"
server_key: "d3f6a3eeb78900f71ec427a7493d12cf"
server_key_iv: "b5fa2a51b1b85085583f95fb696acf84"
server_key_version: "encryptionkey@0"
I20260929 02:52:47.475415 14358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:47.499555 14358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:47.503746 14358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:47.504046 14358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:47.504987 14358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:47.506335 14358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:47.506410 14358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:47.506525 14358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:47.506577 14358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:47.535579 14358 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:34511
I20260929 02:52:47.535630 14485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:34511 every 8 connection(s)
I20260929 02:52:47.536819 14358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:47.537468 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14358
I20260929 02:52:47.537649 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:47.538158 8420 external_mini_cluster.cc:1428] Setting key f9dc89c49da32add34ee0d8d631738e5
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:52:47.555770 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.539302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:58541 (local address 127.8.57.62:46789)
0929 02:52:47.539460 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.539464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.540765 (+ 1301us) server_negotiation.cc:407] Connection header received
0929 02:52:47.542653 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.542668 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.542750 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.542830 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.544313 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.545053 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.546570 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.546965 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.547069 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.550344 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.550349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.550352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.550385 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.552874 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.553622 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.553626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.553629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.553747 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.554146 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.554150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.554152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.554410 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.554536 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.555199 (+ 663us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.555403 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:52:47.556660 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.539845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46789 (local address 127.8.57.3:58541)
0929 02:52:47.540629 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:47.540667 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:47.542407 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:47.543064 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:47.543072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:47.543556 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:47.544154 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:47.544180 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.545197 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.545201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:47.546406 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:47.546417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.547179 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.547186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:47.547435 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.548004 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:47.548026 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:47.550145 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:47.552993 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:47.553002 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:47.553019 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:47.553506 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:47.553845 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:47.553850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:47.553854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:47.554037 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:47.554534 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:47.554542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:47.554845 (+ 303us) client_negotiation.cc:769] Sending connection context
0929 02:52:47.555075 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 02:52:47.555576 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":321,"threads_started":1}
I20260929 02:52:47.557852 14486 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46789
I20260929 02:52:47.558152 14486 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:47.558805 14486 heartbeater.cc:511] Master 127.8.57.62:46789 requested a full tablet report, sending...
I20260929 02:52:47.560123 14030 ts_manager.cc:194] Registered new tserver with Master: bc7731a49d3447d8a5442fb81458e835 (127.8.57.3:34511)
I20260929 02:52:47.560884 14030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:58541
I20260929 02:52:47.565145 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/test-admin.keytab.
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.588559 8420 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 02:52:47.592941 14493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:47.590906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.8.57.1:35677)
0929 02:52:47.591567 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.591574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.591601 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:52:47.591662 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.591670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.591918 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.592042 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.592704 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44762: BlockingRecv error: recv got EOF from 127.0.0.1:44762 (error 108)
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":429,"threads_started":1}
W20260929 02:52:47.593159 8420 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.594748 8420 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.596223 8420 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:47.611843 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.598075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.8.57.1:35677)
0929 02:52:47.598253 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.598259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.598523 (+ 264us) server_negotiation.cc:407] Connection header received
0929 02:52:47.598731 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.598735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.598798 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.598879 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.600144 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.601220 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.602919 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.603450 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.603541 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.606868 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.606878 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.606891 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.606920 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.609486 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.610070 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.610077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.610079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.610192 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.610537 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.610540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.610542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.610861 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.611089 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.611347 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.611564 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 02:52:47.617091 14493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:47.615543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.8.57.1:35677)
0929 02:52:47.615724 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.615730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.616014 (+ 284us) server_negotiation.cc:407] Connection header received
0929 02:52:47.616147 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.616150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.616218 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.616290 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.616911 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44776: BlockingRecv error: recv got EOF from 127.0.0.1:44776 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 02:52:47.617265 8420 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.618631 8420 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.619793 8420 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:47.628940 14219 heartbeater.cc:503] Master 127.8.57.62:46789 was elected leader, sending a full tablet report...
I20260929 02:52:47.632925 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.621075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.8.57.1:35677)
0929 02:52:47.621245 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.621251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.621417 (+ 166us) server_negotiation.cc:407] Connection header received
0929 02:52:47.621585 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.621590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.621650 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.621722 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.622614 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.623310 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.624557 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.624993 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.625105 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.628180 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.628184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.628188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.628225 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.630776 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.631335 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.631339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.631341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.631447 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.631784 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.631787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.631789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.632074 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.632372 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.632477 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.632648 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 02:52:47.636799 14493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:52:47.635282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.8.57.1:35677)
0929 02:52:47.635456 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.635462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.635673 (+ 211us) server_negotiation.cc:407] Connection header received
0929 02:52:47.635800 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.635804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.635886 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.635958 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.636614 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44784: BlockingRecv error: recv got EOF from 127.0.0.1:44784 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 02:52:47.636921 8420 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650321430198-8420-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.638276 8420 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:47.639371 8420 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:52:47.652621 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.640546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.8.57.1:35677)
0929 02:52:47.640721 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.640726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.640928 (+ 202us) server_negotiation.cc:407] Connection header received
0929 02:52:47.641072 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.641076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.641130 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.641201 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.642112 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.642823 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.644026 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.644412 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.644514 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.647693 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.647697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.647700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.647731 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.650271 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.650863 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.650868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.650871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.650988 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.651314 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.651319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.651323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.651583 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.651858 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.652133 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.652331 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:52:47.654057 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14091
I20260929 02:52:47.664574 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.654761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.8.57.1:35677)
0929 02:52:47.654904 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.654908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.655151 (+ 243us) server_negotiation.cc:407] Connection header received
0929 02:52:47.655279 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.655283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.655332 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.655397 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.656326 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.656966 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.658081 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.658483 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.658566 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.660359 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.660363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.660366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.660393 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.662597 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.663148 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.663151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.663153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.663255 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.663542 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.663545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.663547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.663764 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.663994 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.664190 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.664337 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:52:47.665128 14214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:47.746049 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14091
I20260929 02:52:47.758224 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14225
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:52:47.772320 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.759050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.8.57.2:35955)
0929 02:52:47.759596 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.759602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.759625 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:52:47.759683 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.759689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.759910 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.760031 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.760920 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.761893 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.763045 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.763630 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.763723 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.767221 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.767231 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.767244 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.767275 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.769905 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.770493 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.770500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.770502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.770622 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.771014 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.771017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.771019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.771328 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.771548 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.771750 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.771972 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":292,"threads_started":1}
I20260929 02:52:47.773541 14348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:47.855116 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14225
I20260929 02:52:47.867007 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14358
Sep 29 02:52:47 dist-test-slave-1dkt krb5kdc[13985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:52:47.882678 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:47.867958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.8.57.3:34511)
0929 02:52:47.868613 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:47.868620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:47.868650 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:52:47.868714 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:47.868720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:47.868939 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:47.869091 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:47.870257 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.871401 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.872680 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:47.873248 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:47.873373 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:47.877277 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:47.877288 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:47.877303 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:47.877339 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:47.880015 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.880658 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.880668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.880671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.880796 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:47.881155 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:47.881166 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:47.881169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:47.881516 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:47.881825 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:47.882120 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 02:52:47.882391 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":328,"threads_started":1}
I20260929 02:52:47.883963 14481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:47.965418 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14358
I20260929 02:52:47.977411 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 13998
I20260929 02:52:47.978531 14060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:48.079613 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 13998
2026-09-29T02:52:48Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3652 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:48 dist-test-slave-1dkt krb5kdc[14513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:48 dist-test-slave-1dkt krb5kdc[14513](info): set up 2 sockets
Sep 29 02:52:48 dist-test-slave-1dkt krb5kdc[14513](info): commencing operation
krb5kdc: starting...
W20260929 02:52:49.072371 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:52:49 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:52:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:52:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab.
Entry for principal oryx/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab.
I20260929 02:52:49.262593 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:34705
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab
--principal=oryx/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:34705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:49.576123 14526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:49.576403 14526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:49.576522 14526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:49.583645 14526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:49.583752 14526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:49.583820 14526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:49.583911 14526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:49.583968 14526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:49.593569 14526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:34705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab
--principal=oryx/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:49.594838 14526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:49.596144 14526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:49.607013 14532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:49.606985 14531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:49.607041 14534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:49.607725 14526 server_base.cc:1034] running on GCE node
I20260929 02:52:49.608515 14526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:49.609929 14526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:49.611125 14526 hybrid_clock.cc:648] HybridClock initialized: now 1790650369611077 us; error 42 us; skew 500 ppm
I20260929 02:52:49.611445 14526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:49 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:49.618979 14526 init.cc:377] Logged in from keytab as oryx/127.8.57.62@KRBTEST.COM (short username oryx)
I20260929 02:52:49.620570 14526 webserver.cc:460] Webserver started at http://127.8.57.62:45849/ using document root <none> and password file <none>
I20260929 02:52:49.621243 14526 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:49.621342 14526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:49.621605 14526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:49.624679 14526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "91a711a321ce416d941d97b5da125e6a"
format_stamp: "Formatted at 2026-09-29 02:52:49 on dist-test-slave-1dkt"
server_key: "e5158f167dd0d3c336bc250e1957ccae"
server_key_iv: "f044a4a6c03f51e1423d595ccb84088c"
server_key_version: "encryptionkey@0"
I20260929 02:52:49.625447 14526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "91a711a321ce416d941d97b5da125e6a"
format_stamp: "Formatted at 2026-09-29 02:52:49 on dist-test-slave-1dkt"
server_key: "e5158f167dd0d3c336bc250e1957ccae"
server_key_iv: "f044a4a6c03f51e1423d595ccb84088c"
server_key_version: "encryptionkey@0"
I20260929 02:52:49.630528 14526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:49.633854 14541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:49.635250 14526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:52:49.635416 14526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "91a711a321ce416d941d97b5da125e6a"
format_stamp: "Formatted at 2026-09-29 02:52:49 on dist-test-slave-1dkt"
server_key: "e5158f167dd0d3c336bc250e1957ccae"
server_key_iv: "f044a4a6c03f51e1423d595ccb84088c"
server_key_version: "encryptionkey@0"
I20260929 02:52:49.635617 14526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:49.649134 14526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:49.653447 14526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:49.653730 14526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:49.670337 14526 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:34705
I20260929 02:52:49.670344 14592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:34705 every 8 connection(s)
I20260929 02:52:49.671617 14526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:52:49.675203 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:49.680027 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14526
I20260929 02:52:49.680212 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:52:49.680831 8420 external_mini_cluster.cc:1428] Setting key cf3fa53c57faf9e91c960f24337de684
I20260929 02:52:49.685521 14593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Bootstrap starting.
I20260929 02:52:49.689054 14593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:49.690274 14593 log.cc:824] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:49.693542 14593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: No bootstrap required, opened a new log
Sep 29 02:52:49 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:52:49.701424 14593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } }
I20260929 02:52:49.701802 14593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:49.701927 14593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a711a321ce416d941d97b5da125e6a, State: Initialized, Role: FOLLOWER
I20260929 02:52:49.702544 14593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } }
I20260929 02:52:49.702766 14593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:52:49.702893 14593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:52:49.703056 14593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:49.703092 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:49.682678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.8.57.62:34705)
0929 02:52:49.683482 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:49.683496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:49.683538 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:52:49.685215 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:49.685238 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:49.685626 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:49.685983 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:49.687690 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:49.688979 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:49.690645 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:49.691687 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:49.691812 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:49.696163 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:49.696182 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:49.696197 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:49.696231 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:49.699138 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:49.699778 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:49.699787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:49.699794 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:49.699993 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:49.700358 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:49.700363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:49.700366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:49.700886 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:49.701140 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:49.701574 (+ 434us) server_negotiation.cc:299] Negotiation successful
0929 02:52:49.702008 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":360,"threads_started":1}
I20260929 02:52:49.705971 14593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } }
I20260929 02:52:49.706525 14593 leader_election.cc:302] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91a711a321ce416d941d97b5da125e6a; no voters:
I20260929 02:52:49.707082 14593 leader_election.cc:288] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:52:49.707201 14598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:49.708220 14598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 LEADER]: Becoming Leader. State: Replica: 91a711a321ce416d941d97b5da125e6a, State: Running, Role: LEADER
I20260929 02:52:49.708774 14598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } }
I20260929 02:52:49.709234 14593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:49.711258 14599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } }
I20260929 02:52:49.711551 14599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:49.711978 14600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a711a321ce416d941d97b5da125e6a. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } }
I20260929 02:52:49.712313 14600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:49.712445 14605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:49.717628 14605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:49.725656 14605 catalog_manager.cc:1347] Generated new cluster ID: 7438bba307764cdc930a983764e22d2c
I20260929 02:52:49.725777 14605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:49.745558 14605 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:52:49.746892 14605 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:49.755364 14605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Generated new TSK 0
I20260929 02:52:49.756448 14605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab.
Entry for principal oryx/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab.
I20260929 02:52:49.805436 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34705
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:50.120584 14619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:50.120869 14619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:50.120995 14619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:50.128434 14619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:50.128569 14619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:50.128724 14619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:50.138717 14619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:50.140105 14619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:50.141393 14619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:50.154011 14625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:50.154032 14624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:50.154165 14627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:50.154912 14619 server_base.cc:1034] running on GCE node
I20260929 02:52:50.155499 14619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:50.156472 14619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:50.157670 14619 hybrid_clock.cc:648] HybridClock initialized: now 1790650370157617 us; error 49 us; skew 500 ppm
I20260929 02:52:50.157953 14619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:50 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650370, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:50.160933 14619 init.cc:377] Logged in from keytab as oryx/127.8.57.1@KRBTEST.COM (short username oryx)
I20260929 02:52:50.162245 14619 webserver.cc:460] Webserver started at http://127.8.57.1:40773/ using document root <none> and password file <none>
I20260929 02:52:50.162915 14619 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:50.163021 14619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:50.163290 14619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:50.166386 14619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "1dd686a3ff2b4e2ab287add4507b6de3"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "0ec509431466729002a029e3cdc3c03b"
server_key_iv: "063a23c624107b2defac50ddc1846e5a"
server_key_version: "encryptionkey@0"
I20260929 02:52:50.167114 14619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "1dd686a3ff2b4e2ab287add4507b6de3"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "0ec509431466729002a029e3cdc3c03b"
server_key_iv: "063a23c624107b2defac50ddc1846e5a"
server_key_version: "encryptionkey@0"
I20260929 02:52:50.172168 14619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 02:52:50.175455 14634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:50.176802 14619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:50.176950 14619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "1dd686a3ff2b4e2ab287add4507b6de3"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "0ec509431466729002a029e3cdc3c03b"
server_key_iv: "063a23c624107b2defac50ddc1846e5a"
server_key_version: "encryptionkey@0"
I20260929 02:52:50.177129 14619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:50.207458 14619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:50.211814 14619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:50.212127 14619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:50.213145 14619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:50.214496 14619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:50.214574 14619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:50.214694 14619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:50.214753 14619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:50.243666 14619 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:34839
I20260929 02:52:50.243717 14746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:34839 every 8 connection(s)
I20260929 02:52:50.244958 14619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:50.254433 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14619
I20260929 02:52:50.254622 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:52:50.255250 8420 external_mini_cluster.cc:1428] Setting key 24ef23693e4c58ba288a03c9e7e9ea11
Sep 29 02:52:50 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650370, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:52:50.265708 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:50.247547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:50849 (local address 127.8.57.62:34705)
0929 02:52:50.248616 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:50.248621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:50.249157 (+ 536us) server_negotiation.cc:407] Connection header received
0929 02:52:50.250977 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:50.250981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:50.251048 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:50.251151 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:50.252716 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.253444 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.255346 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.255796 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.255939 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:50.259838 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:50.259844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:50.259847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:50.259905 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:50.262375 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:50.263322 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:50.263328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:50.263332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:50.263490 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:50.263898 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:50.263903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:50.263906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:50.264276 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:50.264411 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:50.265150 (+ 739us) server_negotiation.cc:299] Negotiation successful
0929 02:52:50.265368 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":862,"threads_started":1}
I20260929 02:52:50.266626 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:50.248192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:50849)
0929 02:52:50.249005 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:50.249050 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:50.250732 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:50.251325 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:50.251333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:50.251868 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:50.252506 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:50.252526 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.253634 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.253639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:50.255185 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:50.255198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.256034 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.256042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:50.256296 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:50.256996 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:50.257018 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:50.259607 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:50.262548 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:50.262558 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:50.262577 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:50.263202 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:50.263584 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:50.263587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:50.263590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:50.263776 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:50.264431 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:50.264440 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:50.264788 (+ 348us) client_negotiation.cc:769] Sending connection context
0929 02:52:50.265034 (+ 246us) client_negotiation.cc:240] Negotiation successful
0929 02:52:50.265447 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":325,"threads_started":1}
I20260929 02:52:50.268131 14747 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34705
I20260929 02:52:50.268415 14747 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:50.269109 14747 heartbeater.cc:511] Master 127.8.57.62:34705 requested a full tablet report, sending...
I20260929 02:52:50.271451 14558 ts_manager.cc:194] Registered new tserver with Master: 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839)
I20260929 02:52:50.273597 14558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:50849
WARNING: no policy specified for oryx/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab.
Entry for principal oryx/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab.
I20260929 02:52:50.299160 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34705
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:50.609759 14753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:50.610090 14753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:50.610239 14753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:50.617668 14753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:50.617785 14753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:50.617933 14753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:50.628010 14753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:50.629287 14753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:50.630620 14753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:50.643591 14758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:50.643625 14761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:50.643646 14759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:50.644342 14753 server_base.cc:1034] running on GCE node
I20260929 02:52:50.645016 14753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:50.646077 14753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:50.647307 14753 hybrid_clock.cc:648] HybridClock initialized: now 1790650370647255 us; error 44 us; skew 500 ppm
I20260929 02:52:50.647655 14753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:50 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650370, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:50.650808 14753 init.cc:377] Logged in from keytab as oryx/127.8.57.2@KRBTEST.COM (short username oryx)
I20260929 02:52:50.652345 14753 webserver.cc:460] Webserver started at http://127.8.57.2:36855/ using document root <none> and password file <none>
I20260929 02:52:50.653040 14753 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:50.653143 14753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:50.653430 14753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:50.656451 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "029c3d91cc13429ab849c1c0dba9397b"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "9cc9ecb15871b4917dc9cd7d6d746157"
server_key_iv: "8f9d235620373ba8760d54befd94e277"
server_key_version: "encryptionkey@0"
I20260929 02:52:50.657203 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "029c3d91cc13429ab849c1c0dba9397b"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "9cc9ecb15871b4917dc9cd7d6d746157"
server_key_iv: "8f9d235620373ba8760d54befd94e277"
server_key_version: "encryptionkey@0"
I20260929 02:52:50.662300 14753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:52:50.665715 14768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:50.667243 14753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 02:52:50.667419 14753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "029c3d91cc13429ab849c1c0dba9397b"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "9cc9ecb15871b4917dc9cd7d6d746157"
server_key_iv: "8f9d235620373ba8760d54befd94e277"
server_key_version: "encryptionkey@0"
I20260929 02:52:50.667620 14753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:50.696699 14753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:50.700911 14753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:50.701202 14753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:50.702148 14753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:50.703433 14753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:50.703519 14753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:50.703636 14753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:50.703688 14753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:50.732869 14753 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:42449
I20260929 02:52:50.732918 14880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:42449 every 8 connection(s)
I20260929 02:52:50.734089 14753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:50.737618 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14753
I20260929 02:52:50.737794 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:52:50.738305 8420 external_mini_cluster.cc:1428] Setting key b6e3c69b725b9ebb57e3e757475e4b7d
Sep 29 02:52:50 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650370, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:52:50.754815 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:50.736531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:43927 (local address 127.8.57.62:34705)
0929 02:52:50.736735 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:50.736739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:50.737801 (+ 1062us) server_negotiation.cc:407] Connection header received
0929 02:52:50.739847 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:50.739852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:50.739966 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:50.740062 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:50.742064 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.742782 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.744817 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.745204 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.745295 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:50.748961 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:50.748967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:50.748971 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:50.749013 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:50.751939 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:50.752617 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:50.752622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:50.752625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:50.752787 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:50.753140 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:50.753144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:50.753147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:50.753467 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:50.753606 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:50.754245 (+ 639us) server_negotiation.cc:299] Negotiation successful
0929 02:52:50.754468 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:52:50.755673 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:50.736948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:43927)
0929 02:52:50.737645 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:50.737684 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:50.739583 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:50.740351 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:50.740363 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:50.741108 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:50.741880 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:50.741905 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.742912 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.742917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:50.744693 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:50.744707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:50.745402 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:50.745408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:50.745675 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:50.746356 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:50.746384 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:50.748728 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:50.752068 (+ 3340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:50.752075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:50.752091 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:50.752512 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:50.752875 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:50.752878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:50.752880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:50.753051 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:50.753564 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:50.753570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:50.753886 (+ 316us) client_negotiation.cc:769] Sending connection context
0929 02:52:50.754144 (+ 258us) client_negotiation.cc:240] Negotiation successful
0929 02:52:50.754543 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":286,"threads_started":1}
I20260929 02:52:50.757072 14881 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34705
I20260929 02:52:50.757351 14881 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:50.758042 14881 heartbeater.cc:511] Master 127.8.57.62:34705 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:52:50.759457 14558 ts_manager.cc:194] Registered new tserver with Master: 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2:42449)
I20260929 02:52:50.760353 14558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:43927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab.
Entry for principal oryx/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab.
I20260929 02:52:50.782291 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34705
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:51.094568 14886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:51.094841 14886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:51.094940 14886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:51.102444 14886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:51.102556 14886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:51.102669 14886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:51.112413 14886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:51.113643 14886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:51.114887 14886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:51.127632 14891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:51.127597 14894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:51.127665 14892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:51.127995 14886 server_base.cc:1034] running on GCE node
I20260929 02:52:51.128897 14886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:51.129787 14886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:51.130988 14886 hybrid_clock.cc:648] HybridClock initialized: now 1790650371130926 us; error 40 us; skew 500 ppm
I20260929 02:52:51.131354 14886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650371, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:51.135370 14886 init.cc:377] Logged in from keytab as oryx/127.8.57.3@KRBTEST.COM (short username oryx)
I20260929 02:52:51.137673 14886 webserver.cc:460] Webserver started at http://127.8.57.3:33661/ using document root <none> and password file <none>
I20260929 02:52:51.138568 14886 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:51.138700 14886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:51.139144 14886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:51.142330 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "2d591eb2be1c4ffd88fafb45fd4db875"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "4a28190931fdef5e342b42ff3ca8cd8c"
server_key_iv: "f14d6c433167622facff5fa1cfd35db9"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.143049 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "2d591eb2be1c4ffd88fafb45fd4db875"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "4a28190931fdef5e342b42ff3ca8cd8c"
server_key_iv: "f14d6c433167622facff5fa1cfd35db9"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.148044 14886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:52:51.151397 14901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:51.152787 14886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:51.152937 14886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "2d591eb2be1c4ffd88fafb45fd4db875"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "4a28190931fdef5e342b42ff3ca8cd8c"
server_key_iv: "f14d6c433167622facff5fa1cfd35db9"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.153128 14886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:51.170107 14886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:51.174170 14886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:51.174448 14886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:51.175359 14886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:51.176718 14886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:51.176792 14886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:51.176913 14886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:51.176971 14886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:51.206367 14886 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:45913
I20260929 02:52:51.206414 15013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:45913 every 8 connection(s)
I20260929 02:52:51.207558 14886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:51.210090 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 14886
I20260929 02:52:51.210255 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:52:51.210690 8420 external_mini_cluster.cc:1428] Setting key 600233231bd7c5741e0168d51682e7a6
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650371, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:52:51.227699 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.210096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:48223 (local address 127.8.57.62:34705)
0929 02:52:51.210304 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.210310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.211682 (+ 1372us) server_negotiation.cc:407] Connection header received
0929 02:52:51.213872 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.213878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.213972 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.214057 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.215636 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.216465 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.218001 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.218394 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.218504 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.221889 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.221894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.221897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.221932 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.224725 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.225470 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.225476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.225480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.225605 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.226023 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.226028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.226031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.226309 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.226436 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.227138 (+ 702us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.227357 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:52:51.228667 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.210713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:48223)
0929 02:52:51.211519 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.211568 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.213592 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.214320 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.214328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.214854 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.215474 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.215492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.216610 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.216614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.217860 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.217872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.218628 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.218636 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.218852 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.219474 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.219502 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.221654 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.224871 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.224879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.224895 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.225349 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.225710 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.225714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.225717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.225906 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.226432 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.226440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.226787 (+ 347us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.227027 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.227544 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":309,"threads_started":1}
I20260929 02:52:51.229866 15014 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34705
I20260929 02:52:51.230122 15014 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:51.230780 15014 heartbeater.cc:511] Master 127.8.57.62:34705 requested a full tablet report, sending...
I20260929 02:52:51.232070 14558 ts_manager.cc:194] Registered new tserver with Master: 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913)
I20260929 02:52:51.232842 14558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:48223
I20260929 02:52:51.238082 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.57.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.61@KRBTEST.COM" created.
I20260929 02:52:51.277046 14747 heartbeater.cc:503] Master 127.8.57.62:34705 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.61.keytab.
Entry for principal oryx/127.8.57.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.61.keytab.
I20260929 02:52:51.283694 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:39775
--webserver_interface=127.8.57.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.61.keytab
--principal=oryx/127.8.57.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:34705,127.8.57.61:39775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:51.594198 15019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:51.594496 15019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:51.594628 15019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:51.601713 15019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:51.601830 15019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:51.601907 15019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:51.601979 15019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:51.602037 15019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:51.612074 15019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:34705,127.8.57.61:39775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.61.keytab
--principal=oryx/127.8.57.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:39775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.57.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
W20260929 02:52:51.612442 15019 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.57.62:34705,127.8.57.61:39775'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 02:52:51.613448 15019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:51.614714 15019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:51.625741 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:51.625685 15027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:51.626289 15025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:51.626499 15019 server_base.cc:1034] running on GCE node
I20260929 02:52:51.627215 15019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:51.628595 15019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:51.629802 15019 hybrid_clock.cc:648] HybridClock initialized: now 1790650371629737 us; error 45 us; skew 500 ppm
I20260929 02:52:51.630106 15019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650371, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:51.633057 15019 init.cc:377] Logged in from keytab as oryx/127.8.57.61@KRBTEST.COM (short username oryx)
I20260929 02:52:51.634436 15019 webserver.cc:460] Webserver started at http://127.8.57.61:38883/ using document root <none> and password file <none>
I20260929 02:52:51.635114 15019 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:51.635213 15019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:51.635499 15019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:51.638464 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/instance:
uuid: "2e0708665e414be9b968e2a1ad9c822a"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "cb9f32b3c351a86b73fd3609a717fe13"
server_key_iv: "23b7fbeb98de2b60659507a3ff888599"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.639246 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal/instance:
uuid: "2e0708665e414be9b968e2a1ad9c822a"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "cb9f32b3c351a86b73fd3609a717fe13"
server_key_iv: "23b7fbeb98de2b60659507a3ff888599"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.644475 15019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260929 02:52:51.648052 15034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:51.649425 15019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:51.649595 15019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
uuid: "2e0708665e414be9b968e2a1ad9c822a"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "cb9f32b3c351a86b73fd3609a717fe13"
server_key_iv: "23b7fbeb98de2b60659507a3ff888599"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.649807 15019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:51.665833 15019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:51.669863 15019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:51.670146 15019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:51.687261 15019 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.61:39775
I20260929 02:52:51.687281 15085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.61:39775 every 8 connection(s)
I20260929 02:52:51.688597 15019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
I20260929 02:52:51.690850 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 15019
I20260929 02:52:51.692548 15086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:51.701090 15086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650371, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.61@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:52:51.713665 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.692929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.8.57.61:39775)
0929 02:52:51.693647 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.693658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.693705 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:52:51.695226 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.695247 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.695639 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.696033 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.697798 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.699118 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.700788 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.701479 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.701615 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.706226 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.706237 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.706250 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.706294 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.709348 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.710115 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.710123 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.710130 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.710356 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.710791 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.710795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.710798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.711372 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.711614 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.712158 (+ 544us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.712609 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":307,"threads_started":1}
I20260929 02:52:51.718559 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.704090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.8.57.62:34705)
0929 02:52:51.704280 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.704286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.705243 (+ 957us) server_negotiation.cc:407] Connection header received
0929 02:52:51.705462 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.705467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.705540 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.705621 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.707089 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.707812 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.709260 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.709688 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.709794 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.713346 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.713350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.713354 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.713386 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.716148 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.716901 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.716907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.716911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.717045 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.717360 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.717363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.717365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.717661 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.717770 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.718178 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.718343 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:52:51.718602 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.704034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:52778)
0929 02:52:51.705083 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.705131 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.705302 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.705704 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.705710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.706571 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.706939 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.706961 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.707959 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.707965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.709121 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.709133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.709853 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.709859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.710037 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.710669 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.710692 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.713152 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.716273 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.716284 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.716292 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.716773 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.717149 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.717153 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.717155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.717287 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.717767 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.717773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.717913 (+ 140us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.718083 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.718318 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":538,"threads_started":1}
I20260929 02:52:51.720089 15086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650371, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.61@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
I20260929 02:52:51.731374 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.720796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60486)
0929 02:52:51.720960 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.720977 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.721066 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.721290 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.721293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.721606 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.721729 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.721736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.722658 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.722666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.723521 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.723531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.724181 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.724186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.724353 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.724846 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.724863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.726723 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.729271 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.729276 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.729281 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.729607 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.729931 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.729935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.729939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.730052 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.730497 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.730501 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.730639 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.730749 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.731011 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:52:51.731530 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.720767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.8.57.61:39775)
0929 02:52:51.720927 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.720931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.721034 (+ 103us) server_negotiation.cc:407] Connection header received
0929 02:52:51.721137 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.721140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.721192 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.721264 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.721833 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.722538 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.723638 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.724056 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.724156 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.726892 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.726896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.726899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.726925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.729159 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.729705 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.729710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.729714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.729837 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.730141 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.730146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.730150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.730397 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.730515 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.730832 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.731165 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":65}
I20260929 02:52:51.736227 15086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Bootstrap starting.
I20260929 02:52:51.739266 15086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:51.740371 15086 log.cc:824] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:51.742859 15086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: No bootstrap required, opened a new log
I20260929 02:52:51.749688 15086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } }
I20260929 02:52:51.750015 15086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:51.750118 15086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0708665e414be9b968e2a1ad9c822a, State: Initialized, Role: FOLLOWER
I20260929 02:52:51.750659 15086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } }
I20260929 02:52:51.751456 15092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } } }
I20260929 02:52:51.751806 15092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:51.752447 15086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: configured and running, proceeding with master startup.
W20260929 02:52:51.760483 15103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:52:51.760605 15103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:52:51.762902 14881 heartbeater.cc:503] Master 127.8.57.62:34705 was elected leader, sending a full tablet report...
I20260929 02:52:51.763185 15019 master_runner.cc:428] Detected that this master 2e0708665e414be9b968e2a1ad9c822a is joining an existing cluster
I20260929 02:52:51.763268 15019 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.57.61:39775
I20260929 02:52:51.786051 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.766902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.8.57.62:34705)
0929 02:52:51.767216 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.767222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.771203 (+ 3981us) server_negotiation.cc:407] Connection header received
0929 02:52:51.771361 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.771366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.771448 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.771541 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.773092 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.774293 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.776436 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.777051 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.777137 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.779639 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.779646 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.779651 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.779688 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.783030 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.783805 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.783811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.783814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.783982 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.784414 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.784419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.784421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.784757 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.784877 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.785453 (+ 576us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.785684 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260929 02:52:51.787432 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.766932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:52792)
0929 02:52:51.771098 (+ 4166us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.771121 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.771250 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.771741 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.771746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.772485 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.772909 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.772922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.774486 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.774491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.776267 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.776282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.777394 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.777400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.777577 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.778336 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.778360 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.779446 (+ 1086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.783233 (+ 3787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.783243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.783247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.783675 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.784116 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.784120 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.784124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.784307 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.784955 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.784959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.785159 (+ 200us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.786648 (+ 1489us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.786977 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4018,"thread_start_us":10009,"threads_started":1}
I20260929 02:52:51.791591 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.767050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.8.57.61:39775)
0929 02:52:51.767224 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.767228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.776936 (+ 9708us) server_negotiation.cc:407] Connection header received
0929 02:52:51.777298 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.777303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.777372 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.777432 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.778869 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.779974 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.782388 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.783052 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.783141 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.785478 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.785484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.785489 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.785521 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.788956 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.789696 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.789701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.789705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.789856 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.790219 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.790224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.790227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.790557 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.790740 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.791029 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.791256 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.8.57.60@KRBTEST.COM; defaulting to no policy
I20260929 02:52:51.792196 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.767472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60492)
0929 02:52:51.776849 (+ 9377us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.776867 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.777187 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.777595 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.777599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.778230 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.778560 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.778573 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.780159 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.780164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.782243 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.782257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.783444 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.783450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.783611 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.784283 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.784305 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.785305 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.789081 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.789086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.789089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.789561 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.789965 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.789970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.789973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.790134 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.790659 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.790663 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.790806 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.791059 (+ 253us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.791427 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9248,"mutex_wait_us":93,"spinlock_wait_cycles":27904,"thread_start_us":3316,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.60@KRBTEST.COM" created.
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.62@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
I20260929 02:52:51.814270 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.794065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.8.57.61:39775)
0929 02:52:51.794242 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.794248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.800573 (+ 6325us) server_negotiation.cc:407] Connection header received
0929 02:52:51.800985 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.800990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.801098 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.801203 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.802543 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.803623 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.804997 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.805455 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.805565 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.808752 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.808757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.808760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.808816 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.811520 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.812315 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.812319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.812321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.812446 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.812795 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.812798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.812800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.813075 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.813204 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.813661 (+ 457us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.813895 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:52:51.814332 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.794076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60496)
0929 02:52:51.800425 (+ 6349us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.800471 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.800746 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.801271 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.801279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.802122 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.802384 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.802398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.803737 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.803743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.804868 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.804880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.805622 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.805628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.805820 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.806504 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.806525 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.808576 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.811643 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.811657 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.811665 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.812203 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.812545 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.812550 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.812554 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.812693 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.813197 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.813206 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.813376 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.813532 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.813917 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6101,"thread_start_us":5965,"threads_started":1}
I20260929 02:52:51.816404 14558 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.57.61:39775: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a"
member_type: NON_VOTER
last_known_addr {
host: "127.8.57.61"
port: 39775
}
attrs {
promote: true
}
}
dest_uuid: "91a711a321ce416d941d97b5da125e6a"
cas_config_opid_index: -1
I20260929 02:52:51.817387 14558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } }
I20260929 02:52:51.821358 15061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:51.821131 15116 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.60.keytab.
Entry for principal oryx/127.8.57.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.60.keytab.
I20260929 02:52:51.825425 15061 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Refusing update from remote peer 91a711a321ce416d941d97b5da125e6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 02:52:51.826642 15116 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.57.61:39775
I20260929 02:52:51.826776 15117 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:51.827189 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:46531
--webserver_interface=127.8.57.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.60.keytab
--principal=oryx/127.8.57.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:52:51.827325 15019 master.cc:537] Master@127.8.57.61:39775 shutting down...
W20260929 02:52:51.829706 14545 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 02:52:51.836270 14545 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a -> Peer 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61:39775): Couldn't send request to peer 2e0708665e414be9b968e2a1ad9c822a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 02:52:51.838295 15115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:51.838616 15019 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:51.838675 15115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:51.839020 15019 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:51.839224 15019 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: stopping tablet replica
I20260929 02:52:51.844971 15019 master.cc:552] Master@127.8.57.61:39775 shutdown complete.
I20260929 02:52:51.845054 15019 master_runner.cc:305] Clearing existing system tablet
I20260929 02:52:51.845407 15019 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:51.845486 15019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:51.847632 15019 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 02:52:51.849027 15123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:51.849327 15019 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:51.849577 15019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 02:52:51.849663 15019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
uuid: "2e0708665e414be9b968e2a1ad9c822a"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "cb9f32b3c351a86b73fd3609a717fe13"
server_key_iv: "23b7fbeb98de2b60659507a3ff888599"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.853057 15019 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:51.858003 15019 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 02:52:51.858160 15019 log.cc:1196] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 02:52:51.858525 15019 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Deleting consensus metadata
I20260929 02:52:51.858737 15019 master_runner.cc:315] Copying system tablet from 127.8.57.62:34705
I20260929 02:52:51.860914 15019 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: tablet copy: Beginning tablet copy session from remote peer at address 127.8.57.62:34705
I20260929 02:52:51.878863 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.861700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.8.57.62:34705)
0929 02:52:51.861941 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:51.861948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:51.868389 (+ 6441us) server_negotiation.cc:407] Connection header received
0929 02:52:51.868548 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:51.868553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:51.868645 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:51.868731 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:51.869542 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.870225 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.871569 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.872151 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.872262 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.873951 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:51.873955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:51.873959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:51.873990 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:51.876416 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.876992 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.876997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.876999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.877124 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:51.877461 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:51.877466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:51.877469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:51.877754 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:51.877885 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:51.878256 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 02:52:51.878479 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:52:51.879056 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:51.861580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:52798)
0929 02:52:51.868255 (+ 6675us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:51.868278 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:51.868433 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:51.868775 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:51.868780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:51.869190 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:51.869389 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.869398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.870364 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.870370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.871432 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:51.871446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:51.872294 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:51.872299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:51.872470 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:51.873041 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:51.873059 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:51.873745 (+ 686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:51.876538 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.876544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.876548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.876886 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.877230 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:51.877235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:51.877239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:51.877364 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:51.877870 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:51.877874 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:51.878025 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 02:52:51.878148 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 02:52:51.878450 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6536,"thread_start_us":6427,"threads_started":1}
I20260929 02:52:51.880082 14578 tablet_copy_service.cc:140] P 91a711a321ce416d941d97b5da125e6a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2e0708665e414be9b968e2a1ad9c822a ({username='oryx', principal='oryx/127.8.57.61@KRBTEST.COM'} at 127.0.0.1:52798)
I20260929 02:52:51.880344 14578 tablet_copy_service.cc:161] P 91a711a321ce416d941d97b5da125e6a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2e0708665e414be9b968e2a1ad9c822a at {username='oryx', principal='oryx/127.8.57.61@KRBTEST.COM'} at 127.0.0.1:52798: session id = 2e0708665e414be9b968e2a1ad9c822a-00000000000000000000000000000000
I20260929 02:52:51.883407 14578 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 02:52:51.885275 15019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 02:52:51.889842 15019 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: tablet copy: Starting download of 0 data blocks...
I20260929 02:52:51.890113 15019 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: tablet copy: Starting download of 1 WAL segments...
I20260929 02:52:51.892477 15019 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 02:52:51.895524 14578 tablet_copy_service.cc:342] P 91a711a321ce416d941d97b5da125e6a: Request end of tablet copy session 2e0708665e414be9b968e2a1ad9c822a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.57.61@KRBTEST.COM'} at 127.0.0.1:52798
I20260929 02:52:51.895702 14578 tablet_copy_service.cc:433] P 91a711a321ce416d941d97b5da125e6a: ending tablet copy session 2e0708665e414be9b968e2a1ad9c822a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2e0708665e414be9b968e2a1ad9c822a
I20260929 02:52:51.908386 15019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:51.911069 15130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:51.912220 15131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:51.919176 15133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:51.920075 15019 server_base.cc:1034] running on GCE node
I20260929 02:52:51.920413 15019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:51.921049 15019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:51.922258 15019 hybrid_clock.cc:648] HybridClock initialized: now 1790650371922194 us; error 34 us; skew 500 ppm
I20260929 02:52:51.922474 15019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:51 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650371, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:51.924882 15019 init.cc:377] Logged in from keytab as oryx/127.8.57.61@KRBTEST.COM (short username oryx)
I20260929 02:52:51.928798 15019 webserver.cc:460] Webserver started at http://127.8.57.61:35543/ using document root <none> and password file <none>
I20260929 02:52:51.929175 15019 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:51.929263 15019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:51.932468 15019 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:52:51.934521 15140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:51.934933 15019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260929 02:52:51.935083 15019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
uuid: "2e0708665e414be9b968e2a1ad9c822a"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "cb9f32b3c351a86b73fd3609a717fe13"
server_key_iv: "23b7fbeb98de2b60659507a3ff888599"
server_key_version: "encryptionkey@0"
I20260929 02:52:51.935243 15019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:51.966208 15019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:51.970193 15019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:51.991374 15019 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.61:39775
I20260929 02:52:51.991398 15191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.61:39775 every 8 connection(s)
I20260929 02:52:51.992383 15019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
I20260929 02:52:51.997846 15192 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:52:52.001793 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Bootstrap starting.
I20260929 02:52:52.018191 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:52:52.018739 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Bootstrap complete.
I20260929 02:52:52.019788 15192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } }
I20260929 02:52:52.020061 15192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2e0708665e414be9b968e2a1ad9c822a, State: Initialized, Role: LEARNER
I20260929 02:52:52.020319 15192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } }
I20260929 02:52:52.021180 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:52.021580 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: LEARNER
I20260929 02:52:52.021726 15192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:52.037027 15206 catalog_manager.cc:1259] Loaded cluster ID: 7438bba307764cdc930a983764e22d2c
I20260929 02:52:52.037140 15206 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: loading cluster ID for follower catalog manager: success
I20260929 02:52:52.040927 15206 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: acquiring CA information for follower catalog manager: success
I20260929 02:52:52.043658 15206 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:52:52.206208 15120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:52.206486 15120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:52.206614 15120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:52.214511 15120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:52:52.214640 15120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:52.214710 15120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:52.214782 15120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:52:52.214839 15120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:52:52.224892 15120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.60.keytab
--principal=oryx/127.8.57.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:46531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.57.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:52.226234 15120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:52.227478 15120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:52:52.235162 15014 heartbeater.cc:503] Master 127.8.57.62:34705 was elected leader, sending a full tablet report...
W20260929 02:52:52.239179 15211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:52.239125 15212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:52.239094 15214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:52.239910 15120 server_base.cc:1034] running on GCE node
I20260929 02:52:52.240732 15120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:52.242161 15120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:52.243369 15120 hybrid_clock.cc:648] HybridClock initialized: now 1790650372243309 us; error 43 us; skew 500 ppm
I20260929 02:52:52.243698 15120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650372, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:52.246639 15120 init.cc:377] Logged in from keytab as oryx/127.8.57.60@KRBTEST.COM (short username oryx)
I20260929 02:52:52.248087 15120 webserver.cc:460] Webserver started at http://127.8.57.60:41791/ using document root <none> and password file <none>
I20260929 02:52:52.248741 15120 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:52.248854 15120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:52.249119 15120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:52:52.251955 15120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/instance:
uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
format_stamp: "Formatted at 2026-09-29 02:52:52 on dist-test-slave-1dkt"
server_key: "a2a9ea85984be1ba486ed1fc7e1ebaf7"
server_key_iv: "040b4761fd3b8c9d745ca462c06a6b37"
server_key_version: "encryptionkey@0"
I20260929 02:52:52.252681 15120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal/instance:
uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
format_stamp: "Formatted at 2026-09-29 02:52:52 on dist-test-slave-1dkt"
server_key: "a2a9ea85984be1ba486ed1fc7e1ebaf7"
server_key_iv: "040b4761fd3b8c9d745ca462c06a6b37"
server_key_version: "encryptionkey@0"
I20260929 02:52:52.257481 15120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:52:52.260720 15221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:52.261984 15120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:52:52.262149 15120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
format_stamp: "Formatted at 2026-09-29 02:52:52 on dist-test-slave-1dkt"
server_key: "a2a9ea85984be1ba486ed1fc7e1ebaf7"
server_key_iv: "040b4761fd3b8c9d745ca462c06a6b37"
server_key_version: "encryptionkey@0"
I20260929 02:52:52.262320 15120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:52.292070 15120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:52.296243 15120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:52.296500 15120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:52.312620 15120 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.60:46531
I20260929 02:52:52.312644 15272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.60:46531 every 8 connection(s)
I20260929 02:52:52.313903 15120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
I20260929 02:52:52.317523 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:52.318673 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 15120
I20260929 02:52:52.326963 15273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.60@KRBTEST.COM
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650372, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.60@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:52:52.343153 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.320896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.8.57.60:46531)
0929 02:52:52.321793 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.321808 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.321865 (+ 57us) server_negotiation.cc:407] Connection header received
0929 02:52:52.323831 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.323862 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.324423 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.324882 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.326122 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.328070 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.329418 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.330399 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.330572 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.335314 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.335326 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.335339 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.335383 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.338622 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.339624 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.339632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.339642 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.339814 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.340227 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.340232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.340236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.340722 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.340963 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.341409 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.341804 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":409,"threads_started":1}
I20260929 02:52:52.345166 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.329903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.8.57.62:34705)
0929 02:52:52.330101 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.330118 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.330715 (+ 597us) server_negotiation.cc:407] Connection header received
0929 02:52:52.330955 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.330961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.331042 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.331120 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.332204 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.333294 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.334837 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.335384 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.335506 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.339157 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.339164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.339168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.339238 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.341977 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.342718 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.342721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.342723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.342826 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.343171 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.343174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.343176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.344062 (+ 886us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.344194 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.344633 (+ 439us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.344852 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:52:52.345238 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.329812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:52800)
0929 02:52:52.330580 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.330617 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.330823 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.331209 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.331217 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.331788 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.332061 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.332070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.333411 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.333414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.334701 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.334716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.335595 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.335610 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.335784 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.336411 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.336431 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.338998 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.342090 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.342099 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.342105 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.342626 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.342914 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.342919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.342923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.343076 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.344190 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.344197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.344352 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.344520 (+ 168us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.344831 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":372,"threads_started":1}
I20260929 02:52:52.346612 15273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650372, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.60@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
I20260929 02:52:52.358515 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.347263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60504)
0929 02:52:52.347421 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.347436 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.347511 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.348112 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.348117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.348418 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.348543 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.348551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.349765 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.349770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.350665 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.350684 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.351331 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.351337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.351493 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.351991 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.352007 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.353891 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.356542 (+ 2651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.356546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.356548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.356957 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.357304 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.357308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.357311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.357439 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.357852 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.357855 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.357973 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.358074 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.358295 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 02:52:52.358534 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.347344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.8.57.61:39775)
0929 02:52:52.347825 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.347830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.347843 (+ 13us) server_negotiation.cc:407] Connection header received
0929 02:52:52.347919 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.347924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.347982 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.348071 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.348681 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.349650 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.350801 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.351200 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.351307 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.354058 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.354063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.354067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.354098 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.356431 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.357055 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.357059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.357061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.357184 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.357518 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.357521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.357523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.357770 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.357882 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.358140 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.358300 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":260,"threads_started":1}
I20260929 02:52:52.359675 15273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650372, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.60@KRBTEST.COM for oryx/127.8.57.60@KRBTEST.COM
I20260929 02:52:52.370847 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.360419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.0.0.1:33644)
0929 02:52:52.360574 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.360590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.360698 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.360942 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.360946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.361278 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.361405 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.361412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.362296 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.362301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.363112 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.363122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.363765 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.363771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.363993 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.364502 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.364518 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.366497 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.368976 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.368980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.368983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.369313 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.369590 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.369593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.369595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.369726 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.370146 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.370150 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.370263 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.370364 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.370551 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 02:52:52.370937 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.360401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33644 (local address 127.8.57.60:46531)
0929 02:52:52.360567 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.360572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.360645 (+ 73us) server_negotiation.cc:407] Connection header received
0929 02:52:52.360797 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.360800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.360847 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.360915 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.361526 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.362168 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.363242 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.363621 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.363723 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.366671 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.366677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.366681 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.366715 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.368862 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.369408 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.369412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.369414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.369516 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.369805 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.369808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.369810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.370057 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.370155 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.370426 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.370648 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":49}
I20260929 02:52:52.375375 15273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Bootstrap starting.
I20260929 02:52:52.378460 15273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:52.379478 15273 log.cc:824] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:52.382198 15273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: No bootstrap required, opened a new log
I20260929 02:52:52.389106 15273 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } }
I20260929 02:52:52.389475 15273 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:52.389714 15273 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e7ea0a14ca4aed94f44bad47c5fcfd, State: Initialized, Role: FOLLOWER
I20260929 02:52:52.390321 15273 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } }
I20260929 02:52:52.391330 15280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } } }
I20260929 02:52:52.391662 15280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:52.392385 15273 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:52.397598 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14619
W20260929 02:52:52.400820 15291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:52:52.400941 15291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:52:52.405151 15120 master_runner.cc:428] Detected that this master d6e7ea0a14ca4aed94f44bad47c5fcfd is joining an existing cluster
I20260929 02:52:52.405269 15120 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.57.60:46531
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.1@KRBTEST.COM
I20260929 02:52:52.419516 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.398629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.8.57.1:34839)
0929 02:52:52.399247 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.399256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.399284 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:52:52.399354 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.399362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.399606 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.401137 (+ 1531us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.401383 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.402762 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.404656 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.405468 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.405593 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.411259 (+ 5666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.411274 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.411290 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.411333 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.415170 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.416208 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.416218 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.416223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.416389 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.416854 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.416859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.416863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.417310 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.417631 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.418840 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.419133 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":1974,"threads_started":1}
I20260929 02:52:52.420991 14742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:52.438807 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.413550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.8.57.60:46531)
0929 02:52:52.413715 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.413721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.424418 (+ 10697us) server_negotiation.cc:407] Connection header received
0929 02:52:52.424625 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.424630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.424692 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.424749 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.426031 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.427140 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.429001 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.429660 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.429749 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.431966 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.431972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.431976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.432006 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.435550 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.436728 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.436733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.436736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.436876 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.437208 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.437213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.437217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.437638 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.437772 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.438232 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.438438 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 02:52:52.439736 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.413127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.0.0.1:33656)
0929 02:52:52.424319 (+ 11192us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.424341 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.424516 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.424908 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.424914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.425567 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.425869 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.425881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.427342 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.427346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.428866 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.428879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.430037 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.430044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.430192 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.430759 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.430782 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.431777 (+ 995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.435684 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.435689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.435693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.436620 (+ 927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.436979 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.436983 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.436986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.437113 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.437830 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.437834 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.437985 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.439045 (+ 1060us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.439308 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11050,"thread_start_us":10865,"threads_started":1}
I20260929 02:52:52.446882 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.412257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.8.57.62:34705)
0929 02:52:52.412422 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.412428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.412535 (+ 107us) server_negotiation.cc:407] Connection header received
0929 02:52:52.412794 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.412800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.412880 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.412963 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.414461 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.415586 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.417850 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.418516 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.418615 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.428323 (+ 9708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.428330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.428335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.428383 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.431786 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.444365 (+ 12579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.444372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.444376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.444603 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.445080 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.445086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.445090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.445474 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.445611 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.446210 (+ 599us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.446484 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:52:52.447939 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.411766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:52810)
0929 02:52:52.412447 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.412465 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.412644 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.413175 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.413180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.413881 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.414275 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.414287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.415795 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.415800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.417672 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.417687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.419013 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.419020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.419192 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.420148 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.420171 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.427281 (+ 7110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.431972 (+ 4691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.431977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.431981 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.444219 (+ 12238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.444759 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.444765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.444769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.444958 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.445692 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.445697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.445889 (+ 192us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.447122 (+ 1233us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.447435 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"spinlock_wait_cycles":51968,"thread_start_us":535,"threads_started":1}
I20260929 02:52:52.461462 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.419561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.8.57.61:39775)
0929 02:52:52.419735 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.419740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.440052 (+ 20312us) server_negotiation.cc:407] Connection header received
0929 02:52:52.444168 (+ 4116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.444176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.444275 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.444370 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.445373 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.446470 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.450342 (+ 3872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.451733 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.451834 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.454348 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.454356 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.454362 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.454406 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.458100 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.458957 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.458964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.458968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.459138 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.459555 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.459561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.459565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.459987 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.460118 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.460720 (+ 602us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.461047 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:52:52.462639 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.420160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60510)
0929 02:52:52.439962 (+ 19802us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.439981 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.444016 (+ 4035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.444552 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.444557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.445023 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.445202 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.445212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.446645 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.446651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.448029 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.448043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.452173 (+ 4130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.452180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.452364 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.452942 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.452967 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.454117 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.458279 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.458285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.458289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.458819 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.459277 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.459283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.459287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.459446 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.460198 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.460205 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.460385 (+ 180us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.461942 (+ 1557us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.462339 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19687,"thread_start_us":21527,"threads_started":1}
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.62@KRBTEST.COM for oryx/127.8.57.60@KRBTEST.COM
I20260929 02:52:52.476202 15303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.449654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60522)
0929 02:52:52.456318 (+ 6664us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.456338 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.456466 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.460008 (+ 3542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.460013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.460044 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:52.460318 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.460330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.461856 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.461862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.473163 (+ 11301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.473179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.475339 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.475346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.475520 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.475589 (+ 69us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.475669 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.475818 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6530,"thread_start_us":6395,"threads_started":1}
I20260929 02:52:52.476258 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.457476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.8.57.60:46531)
0929 02:52:52.457642 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.457647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.462538 (+ 4891us) server_negotiation.cc:407] Connection header received
0929 02:52:52.462702 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.462707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.462795 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.462885 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.464142 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.464902 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.466312 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.466899 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.467001 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.469859 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.469863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.469867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.469893 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.473361 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.474141 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.474146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.474150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.474277 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.474664 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.474669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.474673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.474999 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.475115 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.475647 (+ 532us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.475857 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:52:52.476958 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.450118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.8.57.61:39775)
0929 02:52:52.450685 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.450691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.456579 (+ 5888us) server_negotiation.cc:407] Connection header received
0929 02:52:52.456643 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.456648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.456737 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.456852 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:52.460488 (+ 3636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.461697 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.473357 (+ 11660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.474733 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.474858 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.475133 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.476493 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.476637 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":1842,"threads_started":1}
I20260929 02:52:52.477073 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.457162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.0.0.1:33660)
0929 02:52:52.462420 (+ 5258us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.462442 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.462566 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.463194 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.463199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.463656 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.463961 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.463973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.465050 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.465055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.466175 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.466187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.467037 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.467042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.467195 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.467720 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.467736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.469698 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.473514 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.473519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.473524 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.474010 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.474400 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.474405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.474409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.474542 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.475210 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.475215 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.475383 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.476425 (+ 1042us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.476690 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5141,"thread_start_us":5020,"threads_started":1}
I20260929 02:52:52.477958 15167 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260929 02:52:52.478401 14558 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.57.60:46531: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
member_type: NON_VOTER
last_known_addr {
host: "127.8.57.60"
port: 46531
}
attrs {
promote: true
}
}
dest_uuid: "91a711a321ce416d941d97b5da125e6a"
cas_config_opid_index: 5
I20260929 02:52:52.478555 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a711a321ce416d941d97b5da125e6a. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:52.478770 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: LEARNER
W20260929 02:52:52.478992 15302 consensus_queue.cc:1159] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2e0708665e414be9b968e2a1ad9c822a" responder_term: 1 status { last_received { term: 1 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 1 index: 5 } }
I20260929 02:52:52.479408 14558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } }
I20260929 02:52:52.480985 15302 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.486114 15248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:52.488318 15167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 LEARNER]: Refusing update from remote peer 91a711a321ce416d941d97b5da125e6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 02:52:52.490437 15248 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Refusing update from remote peer 91a711a321ce416d941d97b5da125e6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 02:52:52.500047 15306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 02:52:52.501842 15308 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:52.503398 15302 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.57.60:46531
I20260929 02:52:52.504097 15120 master.cc:537] Master@127.8.57.60:46531 shutting down...
I20260929 02:52:52.505479 15307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.505841 15307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:52.516594 15248 raft_consensus.cc:858] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } }
I20260929 02:52:52.516250 15167 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
W20260929 02:52:52.516948 15248 raft_consensus.cc:1565] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } }
I20260929 02:52:52.521905 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.522207 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: LEARNER
I20260929 02:52:52.523262 15306 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: attempting to promote NON_VOTER 2e0708665e414be9b968e2a1ad9c822a to VOTER
I20260929 02:52:52.524214 15306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } }
I20260929 02:52:52.528154 15167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 LEARNER]: Refusing update from remote peer 91a711a321ce416d941d97b5da125e6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 02:52:52.528764 15306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 02:52:52.531389 15302 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.532085 15166 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.535319 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.535637 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:52.535822 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.535914 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.536085 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:52.536130 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
W20260929 02:52:52.541110 14542 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a -> Peer d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60:46531): Couldn't send request to peer d6e7ea0a14ca4aed94f44bad47c5fcfd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
W20260929 02:52:52.552166 15227 tablet.cc:2369] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:52.557825 15248 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:52.562222 15317 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:52.562533 15280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a711a321ce416d941d97b5da125e6a. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:52.562760 15280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 02:52:52.563026 15314 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: NON_VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: true } } }
I20260929 02:52:52.563333 15314 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 02:52:52.562671 15317 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 02:52:52.565871 15120 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260929 02:52:52.566336 15120 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260929 02:52:52.566507 15120 tablet_replica.cc:330] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: stopping tablet replica
I20260929 02:52:52.575078 15120 master.cc:552] Master@127.8.57.60:46531 shutdown complete.
I20260929 02:52:52.575166 15120 master_runner.cc:305] Clearing existing system tablet
I20260929 02:52:52.575542 15120 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:52.575645 15120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:52.578795 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14619
I20260929 02:52:52.584517 15120 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.003s
I20260929 02:52:52.588199 15319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:52.588536 15120 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:52.588792 15120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:52:52.588886 15120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
format_stamp: "Formatted at 2026-09-29 02:52:52 on dist-test-slave-1dkt"
server_key: "a2a9ea85984be1ba486ed1fc7e1ebaf7"
server_key_iv: "040b4761fd3b8c9d745ca462c06a6b37"
server_key_version: "encryptionkey@0"
I20260929 02:52:52.593528 15120 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:52.598611 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:34839
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:34705
--webserver_port=40773
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:52:52.600358 15120 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 02:52:52.600512 15120 log.cc:1196] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 02:52:52.600968 15120 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Deleting consensus metadata
I20260929 02:52:52.601183 15120 master_runner.cc:315] Copying system tablet from 127.8.57.62:34705
I20260929 02:52:52.606061 15120 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: tablet copy: Beginning tablet copy session from remote peer at address 127.8.57.62:34705
I20260929 02:52:52.622167 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.606972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:52824)
0929 02:52:52.607932 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:52.607951 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:52.608169 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:52.608682 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:52.608686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:52.609204 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:52.609458 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.609469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.610865 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.610870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.612310 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:52.612324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.613412 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.613417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:52.613647 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.614353 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:52.614374 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:52.615509 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:52.619390 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.619396 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.619400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.619927 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.620342 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:52.620347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:52.620351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:52.620515 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:52.621094 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:52.621099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:52.621274 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 02:52:52.621412 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 02:52:52.621714 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":630,"threads_started":1}
I20260929 02:52:52.622262 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:52.607264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.8.57.62:34705)
0929 02:52:52.607500 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:52.607506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:52.608039 (+ 533us) server_negotiation.cc:407] Connection header received
0929 02:52:52.608322 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:52.608328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:52.608403 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:52.608473 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:52.609616 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.610698 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.612460 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:52.613011 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:52.613099 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:52.615707 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:52.615714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:52.615719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:52.615757 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:52.619253 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.620056 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.620062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.620066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.620228 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:52.620629 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:52.620634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:52.620638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:52.620978 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:52.621146 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:52.621577 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 02:52:52.621848 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260929 02:52:52.623219 14578 tablet_copy_service.cc:140] P 91a711a321ce416d941d97b5da125e6a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d6e7ea0a14ca4aed94f44bad47c5fcfd ({username='oryx', principal='oryx/127.8.57.60@KRBTEST.COM'} at 127.0.0.1:52824)
I20260929 02:52:52.623384 14578 tablet_copy_service.cc:161] P 91a711a321ce416d941d97b5da125e6a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d6e7ea0a14ca4aed94f44bad47c5fcfd at {username='oryx', principal='oryx/127.8.57.60@KRBTEST.COM'} at 127.0.0.1:52824: session id = d6e7ea0a14ca4aed94f44bad47c5fcfd-00000000000000000000000000000000
I20260929 02:52:52.627022 14578 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 02:52:52.628876 15120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 02:52:52.635435 15120 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: tablet copy: Starting download of 0 data blocks...
I20260929 02:52:52.635723 15120 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: tablet copy: Starting download of 1 WAL segments...
I20260929 02:52:52.638016 15120 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 02:52:52.642267 14578 tablet_copy_service.cc:342] P 91a711a321ce416d941d97b5da125e6a: Request end of tablet copy session d6e7ea0a14ca4aed94f44bad47c5fcfd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.57.60@KRBTEST.COM'} at 127.0.0.1:52824
I20260929 02:52:52.642422 14578 tablet_copy_service.cc:433] P 91a711a321ce416d941d97b5da125e6a: ending tablet copy session d6e7ea0a14ca4aed94f44bad47c5fcfd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d6e7ea0a14ca4aed94f44bad47c5fcfd
I20260929 02:52:52.653481 15120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:52.656260 15327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:52.657948 15328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:52.664412 15330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:52.672719 15120 server_base.cc:1034] running on GCE node
I20260929 02:52:52.673240 15120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:52.689790 15120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:52.691114 15120 hybrid_clock.cc:648] HybridClock initialized: now 1790650372691069 us; error 57 us; skew 500 ppm
I20260929 02:52:52.691383 15120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:52 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650372, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:52.693804 15120 init.cc:377] Logged in from keytab as oryx/127.8.57.60@KRBTEST.COM (short username oryx)
I20260929 02:52:52.696857 15120 webserver.cc:460] Webserver started at http://127.8.57.60:44857/ using document root <none> and password file <none>
I20260929 02:52:52.697173 15120 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:52.697281 15120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:52.700146 15120 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260929 02:52:52.701942 15337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:52.702375 15120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260929 02:52:52.702497 15120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
format_stamp: "Formatted at 2026-09-29 02:52:52 on dist-test-slave-1dkt"
server_key: "a2a9ea85984be1ba486ed1fc7e1ebaf7"
server_key_iv: "040b4761fd3b8c9d745ca462c06a6b37"
server_key_version: "encryptionkey@0"
I20260929 02:52:52.702665 15120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:52.734159 15120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:52.738197 15120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:52.772956 15120 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.60:46531
I20260929 02:52:52.773025 15388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.60:46531 every 8 connection(s)
I20260929 02:52:52.773905 15120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
I20260929 02:52:52.779037 15389 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:52:52.783181 15389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Bootstrap starting.
I20260929 02:52:52.805377 15389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:52:52.805953 15389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: Bootstrap complete.
I20260929 02:52:52.806991 15389 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } }
I20260929 02:52:52.807283 15389 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6e7ea0a14ca4aed94f44bad47c5fcfd, State: Initialized, Role: LEARNER
I20260929 02:52:52.807530 15389 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } }
I20260929 02:52:52.808307 15392 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:52.808691 15389 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:52:52.808935 15392 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: LEARNER
I20260929 02:52:52.824225 15403 catalog_manager.cc:1259] Loaded cluster ID: 7438bba307764cdc930a983764e22d2c
I20260929 02:52:52.824323 15403 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: loading cluster ID for follower catalog manager: success
I20260929 02:52:52.826967 15403 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: acquiring CA information for follower catalog manager: success
I20260929 02:52:52.828784 15403 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:52:52.976759 15324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:52.977054 15324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:52.977195 15324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:52.984584 15324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:52.984719 15324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:52.984867 15324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:52:52.994642 15324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:34839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=40773
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:52.995950 15324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:52.997211 15324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:53.009907 15408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:53.009881 15411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:53.010059 15409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:53.011370 15324 server_base.cc:1034] running on GCE node
I20260929 02:52:53.012035 15324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:53.012965 15324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:53.014156 15324 hybrid_clock.cc:648] HybridClock initialized: now 1790650373014093 us; error 58 us; skew 500 ppm
I20260929 02:52:53.014468 15324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:53.017728 15324 init.cc:377] Logged in from keytab as oryx/127.8.57.1@KRBTEST.COM (short username oryx)
I20260929 02:52:53.019042 15324 webserver.cc:460] Webserver started at http://127.8.57.1:40773/ using document root <none> and password file <none>
I20260929 02:52:53.019703 15324 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:53.019807 15324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:53.025123 15324 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:52:53.027603 15418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:53.028990 15324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:53.029122 15324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "1dd686a3ff2b4e2ab287add4507b6de3"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "0ec509431466729002a029e3cdc3c03b"
server_key_iv: "063a23c624107b2defac50ddc1846e5a"
server_key_version: "encryptionkey@0"
I20260929 02:52:53.030076 15324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:53.045164 15324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:53.049196 15324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:53.049414 15324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:53.050271 15324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:53.051532 15324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:53.051620 15324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:53.051692 15324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:53.051746 15324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:53.080533 15324 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:34839
I20260929 02:52:53.080592 15530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:34839 every 8 connection(s)
I20260929 02:52:53.081971 15324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:52:53.085078 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 15324
I20260929 02:52:53.085440 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14753
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
I20260929 02:52:53.123373 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.086020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:42471 (local address 127.8.57.62:34705)
0929 02:52:53.086246 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.086252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.094550 (+ 8298us) server_negotiation.cc:407] Connection header received
0929 02:52:53.101730 (+ 7180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.101738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.101843 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.101920 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.104868 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.106047 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.108704 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.109343 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.109445 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.115505 (+ 6060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.115513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.115518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.115561 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.119215 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.120231 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.120236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.120240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.120390 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.121160 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.121166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.121169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.121558 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.121700 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.122726 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.122982 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.2@KRBTEST.COM
I20260929 02:52:53.125777 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.085348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:42471)
0929 02:52:53.094324 (+ 8976us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.094395 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.099439 (+ 5044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.102523 (+ 3084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.102536 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.103541 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:53.104625 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.104653 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.106269 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.106275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.108503 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.108519 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.109814 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.109821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.110129 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.110953 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:53.110982 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:53.114175 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:53.119403 (+ 5228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.119413 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.119435 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.120096 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.120526 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.120530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.120533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.120780 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.121815 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:53.121824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:53.122267 (+ 443us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.123694 (+ 1427us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.124299 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8599,"thread_start_us":23525,"threads_started":1}
I20260929 02:52:53.127521 15532 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34705
I20260929 02:52:53.127914 15532 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:53.128407 15540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.086616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:45913 (local address 127.8.57.61:39775)
0929 02:52:53.110803 (+ 24187us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.110808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.110833 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:52:53.110892 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.110897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.110984 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.111068 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.113033 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.114072 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.115499 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.116271 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.116383 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.120128 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.120135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.120139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.120176 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.123817 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.124806 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.124813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.124817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.124973 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.126168 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.126173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.126176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.126520 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.126658 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.127740 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.128046 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24051,"thread_start_us":25126,"threads_started":1}
I20260929 02:52:53.128876 15532 heartbeater.cc:511] Master 127.8.57.62:34705 requested a full tablet report, sending...
I20260929 02:52:53.129097 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.086454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.8.57.1:45913)
0929 02:52:53.110077 (+ 23623us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.110096 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.110215 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.111958 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.111963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.112568 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:53.112830 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.112839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.114242 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.114247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.115338 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.115353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.116596 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.116602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.116812 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.117375 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:53.117397 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:53.119920 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:53.124016 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.124021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.124026 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.124652 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.125096 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.125101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.125105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.125749 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.126714 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:53.126718 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:53.126871 (+ 153us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.127011 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.127300 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23478,"thread_start_us":7353,"threads_started":1}
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.60@KRBTEST.COM
I20260929 02:52:53.130797 14558 ts_manager.cc:194] Re-registered known tserver with Master: 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839)
I20260929 02:52:53.131995 14558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:42471
I20260929 02:52:53.130887 15533 heartbeater.cc:347] Connected to a master server at 127.8.57.61:39775
I20260929 02:52:53.136576 15533 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:53.138739 15533 heartbeater.cc:511] Master 127.8.57.61:39775 requested a full tablet report, sending...
I20260929 02:52:53.140988 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.086491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.8.57.2:42449)
0929 02:52:53.101470 (+ 14979us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.101479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.113595 (+ 12116us) server_negotiation.cc:407] Connection header received
0929 02:52:53.113726 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.113734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.113998 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.114163 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.116090 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.117621 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.119493 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.120380 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.120473 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.126306 (+ 5833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.126321 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.126336 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.126381 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.130148 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.138081 (+ 7933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.138092 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.138097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.138299 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.138795 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.138801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.138805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.139276 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.139589 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.140234 (+ 645us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.140555 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14805,"thread_start_us":14647,"threads_started":1}
I20260929 02:52:53.141502 15157 ts_manager.cc:194] Registered new tserver with Master: 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839)
I20260929 02:52:53.143249 14876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:53.149899 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.084728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:42933 (local address 127.8.57.60:46531)
0929 02:52:53.100988 (+ 16260us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.100995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.111714 (+ 10719us) server_negotiation.cc:407] Connection header received
0929 02:52:53.113096 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.113102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.113183 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.113277 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.115427 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.116831 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.119243 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.119865 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.119992 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.143567 (+ 23575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.143574 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.143577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.143613 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.147070 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.147911 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.147916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.147920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.148049 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.148461 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.148465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.148469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.148798 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.148941 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.149337 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.149541 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16084,"thread_start_us":16741,"threads_started":1}
I20260929 02:52:53.150015 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.086819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.8.57.1:42933)
0929 02:52:53.111608 (+ 24789us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.111625 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.112968 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.114390 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.114394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.114937 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:53.115257 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.115269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.116986 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.116991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.118544 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.118560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.126316 (+ 7756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.126322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.126520 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.127376 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:53.127396 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:53.143321 (+ 15925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:53.147212 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.147219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.147223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.147765 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.148161 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.148166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.148170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.148343 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.148911 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:53.148916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:53.149086 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.149230 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.149573 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24665,"thread_start_us":24512,"threads_started":1}
I20260929 02:52:53.151000 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.134596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.0.0.1:33676)
0929 02:52:53.135156 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.135173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.135288 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.144862 (+ 9574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.144868 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.144899 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:53.145094 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.145106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.146764 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.146770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.148232 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.148245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.150294 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.150303 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.150471 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.150496 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.150587 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.150719 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":251,"threads_started":1}
I20260929 02:52:53.151070 15531 heartbeater.cc:347] Connected to a master server at 127.8.57.60:46531
I20260929 02:52:53.151223 15531 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:53.151515 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.134753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.8.57.60:46531)
0929 02:52:53.144358 (+ 9605us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.144364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.144382 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:52:53.144433 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.144437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.144502 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.144610 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:53.145280 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.146591 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.148355 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.149494 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.149627 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.149860 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.150906 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.151037 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9481,"thread_start_us":9314,"threads_started":1}
I20260929 02:52:53.151655 15531 heartbeater.cc:511] Master 127.8.57.60:46531 requested a full tablet report, sending...
I20260929 02:52:53.152545 15364 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 02:52:53.152981 15392 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a711a321ce416d941d97b5da125e6a. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:53.153252 15392 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: LEARNER
I20260929 02:52:53.153715 15354 ts_manager.cc:194] Registered new tserver with Master: 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839)
I20260929 02:52:53.239856 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14753
I20260929 02:52:53.253207 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:42449
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:34705
--webserver_port=36855
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:52:53.565723 15546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:53.566090 15546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:53.566237 15546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:53.573642 15546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:53.573757 15546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:53.573879 15546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:52:53.583540 15546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:42449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=36855
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:53.584836 15546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:53.586097 15546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:53.598547 15552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:53.598615 15554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:53.598639 15551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:53.599393 15546 server_base.cc:1034] running on GCE node
I20260929 02:52:53.600029 15546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:53.600939 15546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:53.602131 15546 hybrid_clock.cc:648] HybridClock initialized: now 1790650373602085 us; error 34 us; skew 500 ppm
I20260929 02:52:53.602424 15546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:53.605572 15546 init.cc:377] Logged in from keytab as oryx/127.8.57.2@KRBTEST.COM (short username oryx)
I20260929 02:52:53.606984 15546 webserver.cc:460] Webserver started at http://127.8.57.2:36855/ using document root <none> and password file <none>
I20260929 02:52:53.607630 15546 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:53.607764 15546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:53.613197 15546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:52:53.615810 15561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:53.617170 15546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:52:53.617302 15546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "029c3d91cc13429ab849c1c0dba9397b"
format_stamp: "Formatted at 2026-09-29 02:52:50 on dist-test-slave-1dkt"
server_key: "9cc9ecb15871b4917dc9cd7d6d746157"
server_key_iv: "8f9d235620373ba8760d54befd94e277"
server_key_version: "encryptionkey@0"
I20260929 02:52:53.618213 15546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:53.648911 15546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:53.651373 15542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: NON_VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: true } } }
I20260929 02:52:53.651708 15542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:53.652758 15567 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: attempting to promote NON_VOTER d6e7ea0a14ca4aed94f44bad47c5fcfd to VOTER
I20260929 02:52:53.653544 15567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:53.653820 15546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:53.654109 15546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:53.655405 15364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 LEARNER]: Refusing update from remote peer 91a711a321ce416d941d97b5da125e6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 02:52:53.655462 15167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Refusing update from remote peer 91a711a321ce416d941d97b5da125e6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 02:52:53.656064 15546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:53.655956 15566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 02:52:53.656450 15567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 02:52:53.659574 15567 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.660588 15546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:53.660669 15546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:53.660341 15364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.660751 15546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:53.660799 15546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:53.663111 15567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.663322 15567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:53.663910 15542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.664139 15542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:53.664434 15542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.664646 15542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:53.664789 15167 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.667599 15577 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.667943 15577 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:53.669310 15580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91a711a321ce416d941d97b5da125e6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:53.669586 15580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:53.695647 15546 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:42449
I20260929 02:52:53.695709 15682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:42449 every 8 connection(s)
I20260929 02:52:53.697057 15546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:52:53.701239 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 15546
I20260929 02:52:53.701543 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14886
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650373, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.60@KRBTEST.COM
Sep 29 02:52:53 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.3@KRBTEST.COM
I20260929 02:52:53.747368 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.699812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:37427 (local address 127.8.57.61:39775)
0929 02:52:53.725773 (+ 25961us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.725779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.725798 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:52:53.725864 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.725869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.725964 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.726074 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.729280 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.730446 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.733086 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.733802 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.733938 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.738723 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.738730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.738735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.738776 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.742924 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.744423 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.744428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.744432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.744559 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.745067 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.745073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.745077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.745419 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.745547 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.746766 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.746994 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25814,"thread_start_us":25651,"threads_started":1}
I20260929 02:52:53.749336 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.699894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54687 (local address 127.8.57.62:34705)
0929 02:52:53.726598 (+ 26704us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.726605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.726628 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:52:53.726702 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.726707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.726786 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.726891 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.729272 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.730585 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.736373 (+ 5788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.737015 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.737129 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.741438 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.741445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.741451 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.741494 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.744893 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.745887 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.745893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.745897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.746067 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.746484 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.746488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.746492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.746838 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.746970 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.748735 (+ 1765us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.748966 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26571,"thread_start_us":26420,"threads_started":1}
I20260929 02:52:53.749316 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.700523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:54687)
0929 02:52:53.725144 (+ 24621us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.725163 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.725278 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.727112 (+ 1834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.727118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.727889 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:53.729019 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.729049 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.730801 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.730807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.736192 (+ 5385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.736207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.737377 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.737383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.737567 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.738190 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:53.738213 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:53.741219 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:53.745227 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.745232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.745235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.745749 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.746215 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.746219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.746223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.746360 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.747029 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:53.747034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:53.747174 (+ 140us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.747309 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.747761 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24452,"mutex_wait_us":95,"spinlock_wait_cycles":21760,"thread_start_us":15170,"threads_started":1}
I20260929 02:52:53.749332 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.700523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.8.57.2:37427)
0929 02:52:53.715169 (+ 14646us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.715244 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.718805 (+ 3561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.726444 (+ 7639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.726478 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.727852 (+ 1374us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:53.729019 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.729047 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.730668 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.730674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.732941 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.732954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.734040 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.734047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.734360 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.735149 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:53.735179 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:53.738417 (+ 3238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:53.743591 (+ 5174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.743620 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.743639 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.744296 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.744693 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.744698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.744701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.744955 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.745550 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:53.745558 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:53.745968 (+ 410us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.746266 (+ 298us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.748066 (+ 1800us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14242,"mutex_wait_us":414,"thread_start_us":24159,"threads_started":1}
I20260929 02:52:53.750926 15685 heartbeater.cc:347] Connected to a master server at 127.8.57.61:39775
I20260929 02:52:53.750948 15684 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34705
I20260929 02:52:53.751278 15684 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:53.751281 15685 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:53.752245 15684 heartbeater.cc:511] Master 127.8.57.62:34705 requested a full tablet report, sending...
I20260929 02:52:53.752600 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.701743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:42773 (local address 127.8.57.60:46531)
0929 02:52:53.732288 (+ 30545us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.732295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.734153 (+ 1858us) server_negotiation.cc:407] Connection header received
0929 02:52:53.734221 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.734225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.734313 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.734405 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.735712 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.737936 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.739867 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.740463 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.740567 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.744339 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.744345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.744350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.744389 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.748372 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.749101 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.749107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.749122 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.749255 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.750363 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.750368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.750371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.750703 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.750823 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.752033 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.752251 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30404,"thread_start_us":30285,"threads_started":1}
I20260929 02:52:53.752815 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.701277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.8.57.2:42773)
0929 02:52:53.733907 (+ 32630us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:53.733924 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:53.734044 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:53.734590 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:53.734595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:53.735148 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:53.735518 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.735530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.738120 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.738125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.739712 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:53.739727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.741094 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.741099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:53.741274 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.741724 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:53.741746 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:53.744136 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:53.748514 (+ 4378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.748519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.748522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.748972 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.750049 (+ 1077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:53.750062 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:53.750066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:53.750234 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:53.750894 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:53.750899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:53.751128 (+ 229us) client_negotiation.cc:769] Sending connection context
0929 02:52:53.751290 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 02:52:53.751698 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32499,"thread_start_us":32354,"threads_started":1}
I20260929 02:52:53.753001 15685 heartbeater.cc:511] Master 127.8.57.61:39775 requested a full tablet report, sending...
I20260929 02:52:53.753628 15683 heartbeater.cc:347] Connected to a master server at 127.8.57.60:46531
I20260929 02:52:53.753747 15683 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:53.753959 14558 ts_manager.cc:194] Re-registered known tserver with Master: 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2:42449)
I20260929 02:52:53.754073 15157 ts_manager.cc:194] Registered new tserver with Master: 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2:42449)
I20260929 02:52:53.754127 15683 heartbeater.cc:511] Master 127.8.57.60:46531 requested a full tablet report, sending...
I20260929 02:52:53.755016 14558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:54687
I20260929 02:52:53.755707 15354 ts_manager.cc:194] Registered new tserver with Master: 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2:42449)
I20260929 02:52:53.755836 15694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:53.702439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.8.57.3:45913)
0929 02:52:53.732799 (+ 30360us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:53.732809 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:53.732839 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:52:53.732910 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:53.732920 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:53.733211 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:53.733397 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:53.735299 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.736951 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.739191 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:53.740350 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:53.740472 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:53.746984 (+ 6512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:53.747001 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:53.747018 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:53.747061 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:53.751194 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.752176 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.752185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.752189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.752339 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:53.753286 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:53.753290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:53.753293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:53.753840 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:53.754122 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:53.755205 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0929 02:52:53.755441 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30159,"thread_start_us":29992,"threads_started":1}
I20260929 02:52:53.757263 15009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:53.840097 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14886
I20260929 02:52:53.852996 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:45913
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:34705
--webserver_port=33661
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:52:54.143194 15532 heartbeater.cc:503] Master 127.8.57.62:34705 was elected leader, sending a full tablet report...
W20260929 02:52:54.165314 15696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:52:54.165612 15696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:52:54.165746 15696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:52:54.173316 15696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:52:54.173444 15696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:52:54.173599 15696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:52:54.183435 15696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:45913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=33661
--tserver_master_addrs=127.8.57.62:34705,127.8.57.61:39775,127.8.57.60:46531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:52:54.184841 15696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:52:54.186156 15696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:52:54.199915 15701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:54.199970 15702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:52:54.199944 15704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:52:54.200768 15696 server_base.cc:1034] running on GCE node
I20260929 02:52:54.201350 15696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:52:54.202266 15696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:52:54.203470 15696 hybrid_clock.cc:648] HybridClock initialized: now 1790650374203410 us; error 44 us; skew 500 ppm
I20260929 02:52:54.203776 15696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:52:54 dist-test-slave-1dkt krb5kdc[14513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650374, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:52:54.206884 15696 init.cc:377] Logged in from keytab as oryx/127.8.57.3@KRBTEST.COM (short username oryx)
I20260929 02:52:54.208285 15696 webserver.cc:460] Webserver started at http://127.8.57.3:33661/ using document root <none> and password file <none>
I20260929 02:52:54.208971 15696 fs_manager.cc:359] Metadata directory not provided
I20260929 02:52:54.209098 15696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:52:54.214509 15696 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:52:54.217197 15711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:54.218469 15696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:52:54.218607 15696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "2d591eb2be1c4ffd88fafb45fd4db875"
format_stamp: "Formatted at 2026-09-29 02:52:51 on dist-test-slave-1dkt"
server_key: "4a28190931fdef5e342b42ff3ca8cd8c"
server_key_iv: "f14d6c433167622facff5fa1cfd35db9"
server_key_version: "encryptionkey@0"
I20260929 02:52:54.219543 15696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:52:54.255021 15696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:52:54.258985 15696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:52:54.259251 15696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:52:54.260203 15696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:52:54.261478 15696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:52:54.261561 15696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:54.261667 15696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:52:54.261736 15696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:52:54.289449 15696 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:45913
I20260929 02:52:54.289500 15823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:45913 every 8 connection(s)
I20260929 02:52:54.290813 15696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:52:54.301219 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 15696
Sep 29 02:52:54 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650374, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
Sep 29 02:52:54 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650374, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.61@KRBTEST.COM
Sep 29 02:52:54 dist-test-slave-1dkt krb5kdc[14513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650374, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.60@KRBTEST.COM
I20260929 02:52:54.341732 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.294169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.8.57.3:51233)
0929 02:52:54.315043 (+ 20874us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.315064 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.315186 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.320897 (+ 5711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.320905 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.321596 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:54.322019 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.322035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.323510 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.323515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.325074 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.325091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.326002 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.326008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.326168 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.327025 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:54.327046 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:54.330243 (+ 3197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:54.335255 (+ 5012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:54.335266 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:54.335287 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:54.336063 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:54.337404 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:54.337410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:54.337414 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:54.337675 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:54.338695 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:54.338705 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:54.339245 (+ 540us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.339566 (+ 321us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.340244 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20712,"mutex_wait_us":29,"thread_start_us":26543,"threads_started":1}
I20260929 02:52:54.344120 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.293920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:51233 (local address 127.8.57.61:39775)
0929 02:52:54.320495 (+ 26575us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.320500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.320514 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:52:54.320559 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.320562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.320628 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.320722 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.322206 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.323351 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.325238 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.325830 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.325963 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.330526 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.330532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.330537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.330576 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.335055 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.337073 (+ 2018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.337080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.337084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.337265 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.338071 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.338077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.338081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.338463 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.338598 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.343445 (+ 4847us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.343701 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26483,"thread_start_us":26408,"threads_started":1}
I20260929 02:52:54.345293 15826 heartbeater.cc:347] Connected to a master server at 127.8.57.61:39775
I20260929 02:52:54.345569 15826 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:54.346503 15826 heartbeater.cc:511] Master 127.8.57.61:39775 requested a full tablet report, sending...
I20260929 02:52:54.348475 15157 ts_manager.cc:194] Registered new tserver with Master: 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913)
I20260929 02:52:54.350226 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.294173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:51543)
0929 02:52:54.309896 (+ 15723us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.309953 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.312416 (+ 2463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.316339 (+ 3923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.316354 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.317410 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:54.318365 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.318393 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.320764 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.320774 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.323856 (+ 3082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.323895 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.325158 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.325165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.325484 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.326519 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:54.326557 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:54.330890 (+ 4333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:54.344554 (+ 13664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:54.344558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:54.344562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:54.345050 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:54.347623 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:54.347628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:54.347631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:54.347792 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:54.349211 (+ 1419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:54.349217 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:54.349378 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.349508 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.349801 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14229,"mutex_wait_us":1,"thread_start_us":14320,"threads_started":1}
I20260929 02:52:54.350227 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.293822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:51543 (local address 127.8.57.62:34705)
0929 02:52:54.315575 (+ 21753us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.315582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.315604 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:52:54.315720 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.315725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.315808 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.318584 (+ 2776us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.318770 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.320583 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.324060 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.324699 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.324795 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.331121 (+ 6326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.331128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.331133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.331184 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.344103 (+ 12919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.345203 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.345210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.345214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.347484 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.348664 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.348669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.348672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.349006 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.349139 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.349638 (+ 499us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.349863 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21628,"thread_start_us":21526,"threads_started":1}
I20260929 02:52:54.351672 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.294260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.8.57.3:60763)
0929 02:52:54.321268 (+ 27008us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.321291 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.321447 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.321934 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.321943 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.322563 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:52:54.323218 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.323231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.324740 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.324746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.326825 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.326839 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.328089 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.328097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.328308 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.329007 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:52:54.329029 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:52:54.332545 (+ 3516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:52:54.347962 (+ 15417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:54.347967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:54.347970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:54.348429 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:54.349946 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:52:54.349951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:52:54.349955 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:52:54.350133 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:52:54.350768 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:52:54.350773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:52:54.350899 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.351045 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.351304 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26851,"mutex_wait_us":467,"thread_start_us":20054,"threads_started":1}
I20260929 02:52:54.351701 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.293479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:60763 (local address 127.8.57.60:46531)
0929 02:52:54.320225 (+ 26746us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.320230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.321559 (+ 1329us) server_negotiation.cc:407] Connection header received
0929 02:52:54.321635 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.321641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.321737 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.322706 (+ 969us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.323410 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.324557 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.327003 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.327662 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.327788 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.332766 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.332773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.332778 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.332818 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.347359 (+ 14541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.349647 (+ 2288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.349652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.349654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.349814 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.350286 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.350290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.350293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.350610 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.350804 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.351153 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.351384 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26641,"thread_start_us":26510,"threads_started":1}
I20260929 02:52:54.353006 15825 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34705
I20260929 02:52:54.353150 15825 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:54.353148 15824 heartbeater.cc:347] Connected to a master server at 127.8.57.60:46531
I20260929 02:52:54.353312 15824 heartbeater.cc:464] Registering TS with master...
I20260929 02:52:54.353698 15824 heartbeater.cc:511] Master 127.8.57.60:46531 requested a full tablet report, sending...
I20260929 02:52:54.354811 15354 ts_manager.cc:194] Registered new tserver with Master: 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913)
I20260929 02:52:54.354857 15825 heartbeater.cc:511] Master 127.8.57.62:34705 requested a full tablet report, sending...
I20260929 02:52:54.355790 14558 ts_manager.cc:194] Re-registered known tserver with Master: 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913)
I20260929 02:52:54.356865 14558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:51543
I20260929 02:52:54.362902 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.336236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.8.57.61:39775)
0929 02:52:54.342268 (+ 6032us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.342273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.347554 (+ 5281us) server_negotiation.cc:407] Connection header received
0929 02:52:54.347607 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.347612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.347698 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.347800 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.349986 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.351184 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.353046 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.353636 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.353747 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.356959 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.356966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.356971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.357007 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.360058 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.360860 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.360865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.360868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.361013 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.361487 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.361492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.361495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.361846 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.362080 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.362423 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.362629 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5910,"thread_start_us":5777,"threads_started":1}
I20260929 02:52:54.365581 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.329984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.8.57.62:34705)
0929 02:52:54.346536 (+ 16552us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.346541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.346558 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:54.346605 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.346609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.346671 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.346783 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.351858 (+ 5075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.352829 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.355253 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.355886 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.355996 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.359384 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.359391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.359396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.359431 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.362468 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.363703 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.363708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.363712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.363847 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.364303 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.364308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.364311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.364606 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.364798 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.365104 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.365262 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16430,"thread_start_us":19438,"threads_started":1}
I20260929 02:52:54.367771 15840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.330770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.8.57.60:46531)
0929 02:52:54.345775 (+ 15005us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.345781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.345802 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:52:54.345854 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.345859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.345927 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.346058 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.353560 (+ 7502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.354564 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.359149 (+ 4585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.359695 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.359815 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.362395 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.362401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.362406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.362439 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.365361 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.365970 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.365973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.365976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.366112 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.366456 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.366461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.366464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.366769 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.366997 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.367259 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.367462 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14845,"thread_start_us":18736,"threads_started":1}
I20260929 02:52:54.369560 14558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:54.376271 14558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:54.403630 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.392990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.8.57.1:34839)
0929 02:52:54.393546 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.393556 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.393914 (+ 358us) server_negotiation.cc:407] Connection header received
0929 02:52:54.395287 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.395298 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.395615 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.395797 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:54.396653 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.398579 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.400817 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.402220 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.402353 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.402560 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.403142 (+ 582us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.403322 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":267,"threads_started":1}
I20260929 02:52:54.403664 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.392538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:34839 (local address 127.0.0.1:57932)
0929 02:52:54.393036 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.393052 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.393167 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.396110 (+ 2943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.396116 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.396143 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:54.396472 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.396484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.398738 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.398744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.400670 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.400683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.402690 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.402696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.402893 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.402928 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.403090 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.403223 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":596,"threads_started":1}
I20260929 02:52:54.408078 15856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.394101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:42449 (local address 127.0.0.1:39306)
0929 02:52:54.394693 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.394716 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.395032 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.401066 (+ 6034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.401071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.401098 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:54.401360 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.401371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.403389 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.403393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.405217 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.405233 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.407277 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.407283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.407466 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.407498 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.407574 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.407708 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":612,"threads_started":1}
I20260929 02:52:54.407619 15466 tablet_service.cc:1467] Processing CreateTablet for tablet c4cf016056b44e4e904558518c3d271e (DEFAULT_TABLE table=test-table [id=765d70a4e3b0427faf10b84627e0ac3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:54.408716 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.394576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39306 (local address 127.8.57.2:42449)
0929 02:52:54.400360 (+ 5784us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.400371 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.400402 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:52:54.400514 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.400524 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.400822 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.401088 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:54.401600 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.403231 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.405392 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.406794 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.406926 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.407136 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.408324 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.408493 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5588,"thread_start_us":5465,"threads_started":1}
I20260929 02:52:54.409214 15466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4cf016056b44e4e904558518c3d271e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:54.410990 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.392122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45913 (local address 127.0.0.1:55344)
0929 02:52:54.392668 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.392693 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.392839 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.402460 (+ 9621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.402464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.402487 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:54.402834 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.402844 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.405337 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.405342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.407359 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.407371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.410235 (+ 2864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.410242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.410433 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.410460 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.410538 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.410667 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260929 02:52:54.411032 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.392333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.8.57.3:45913)
0929 02:52:54.401834 (+ 9501us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.401842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.401875 (+ 33us) server_negotiation.cc:407] Connection header received
0929 02:52:54.401969 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.401978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.402236 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.403053 (+ 817us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:54.403391 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.405205 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.408922 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.410051 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.410169 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.410376 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.410594 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.410747 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9345,"thread_start_us":9182,"threads_started":1}
I20260929 02:52:54.414775 15618 tablet_service.cc:1467] Processing CreateTablet for tablet c4cf016056b44e4e904558518c3d271e (DEFAULT_TABLE table=test-table [id=765d70a4e3b0427faf10b84627e0ac3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:54.414775 15759 tablet_service.cc:1467] Processing CreateTablet for tablet c4cf016056b44e4e904558518c3d271e (DEFAULT_TABLE table=test-table [id=765d70a4e3b0427faf10b84627e0ac3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:54.416220 15759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4cf016056b44e4e904558518c3d271e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:54.416961 15618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4cf016056b44e4e904558518c3d271e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:54.421516 15858 tablet_bootstrap.cc:492] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Bootstrap starting.
I20260929 02:52:54.425136 15858 tablet_bootstrap.cc:654] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:54.426330 15858 log.cc:824] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:54.428936 15858 tablet_bootstrap.cc:492] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: No bootstrap required, opened a new log
I20260929 02:52:54.429251 15858 ts_tablet_manager.cc:1397] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 02:52:54.433558 15860 tablet_bootstrap.cc:492] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Bootstrap starting.
I20260929 02:52:54.433769 15861 tablet_bootstrap.cc:492] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Bootstrap starting.
I20260929 02:52:54.436765 15861 tablet_bootstrap.cc:654] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:54.436997 15860 tablet_bootstrap.cc:654] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:54.437876 15861 log.cc:824] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:54.438148 15860 log.cc:824] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Log is configured to *not* fsync() on all Append() calls
I20260929 02:52:54.438485 15858 raft_consensus.cc:348] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.438930 15858 raft_consensus.cc:374] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:54.439035 15858 raft_consensus.cc:729] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd686a3ff2b4e2ab287add4507b6de3, State: Initialized, Role: FOLLOWER
I20260929 02:52:54.439744 15858 consensus_queue.cc:260] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.441465 15858 ts_tablet_manager.cc:1428] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260929 02:52:54.441614 15860 tablet_bootstrap.cc:492] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: No bootstrap required, opened a new log
I20260929 02:52:54.441843 15860 ts_tablet_manager.cc:1397] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:52:54.449558 15861 tablet_bootstrap.cc:492] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: No bootstrap required, opened a new log
I20260929 02:52:54.449859 15861 ts_tablet_manager.cc:1397] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260929 02:52:54.451471 15860 raft_consensus.cc:348] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.451864 15860 raft_consensus.cc:374] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:54.451995 15860 raft_consensus.cc:729] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d591eb2be1c4ffd88fafb45fd4db875, State: Initialized, Role: FOLLOWER
I20260929 02:52:54.452512 15860 consensus_queue.cc:260] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.453541 15825 heartbeater.cc:503] Master 127.8.57.62:34705 was elected leader, sending a full tablet report...
I20260929 02:52:54.454164 15860 ts_tablet_manager.cc:1428] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260929 02:52:54.459668 15861 raft_consensus.cc:348] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.460074 15861 raft_consensus.cc:374] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:54.460150 15861 raft_consensus.cc:729] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 029c3d91cc13429ab849c1c0dba9397b, State: Initialized, Role: FOLLOWER
I20260929 02:52:54.460669 15861 consensus_queue.cc:260] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.461629 15684 heartbeater.cc:503] Master 127.8.57.62:34705 was elected leader, sending a full tablet report...
I20260929 02:52:54.462260 15861 ts_tablet_manager.cc:1428] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260929 02:52:54.544150 15827 tablet.cc:2369] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:52:54.613358 15534 tablet.cc:2369] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:52:54.699759 15686 tablet.cc:2369] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:52:54.786342 15866 raft_consensus.cc:482] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:54.786648 15866 raft_consensus.cc:504] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.788225 15866 leader_election.cc:288] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913), 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839)
I20260929 02:52:54.796132 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.788595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:42111 (local address 127.8.57.1:34839)
0929 02:52:54.788890 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.788896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.789591 (+ 695us) server_negotiation.cc:407] Connection header received
0929 02:52:54.789731 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.789734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.789787 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.789853 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:54.790362 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.791332 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.793842 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.795045 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.795176 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.795277 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.795726 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.795861 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260929 02:52:54.796308 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.788447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45913 (local address 127.8.57.2:45589)
0929 02:52:54.789047 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.789066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.789235 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.789615 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.789620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.789640 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:54.789998 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.790011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.791281 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.791285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.793690 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.793706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.795298 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.795303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.795524 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.795597 (+ 73us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.795694 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.795896 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":226,"thread_start_us":834,"threads_started":1}
I20260929 02:52:54.796452 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.788471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:34839 (local address 127.8.57.2:42111)
0929 02:52:54.789495 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:54.789508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:54.789615 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:54.789900 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:54.789905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:54.789931 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:54.790228 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.790236 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.791442 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.791450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.793700 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:54.793708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.795273 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.795285 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:54.795518 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.795595 (+ 77us) client_negotiation.cc:769] Sending connection context
0929 02:52:54.795700 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 02:52:54.796007 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":144,"thread_start_us":263,"threads_started":1}
I20260929 02:52:54.796492 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.788646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:45589 (local address 127.8.57.3:45913)
0929 02:52:54.789042 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.789047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.789131 (+ 84us) server_negotiation.cc:407] Connection header received
0929 02:52:54.789426 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.789431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.789491 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.789568 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:54.790138 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.791155 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.793842 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.795106 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.795214 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.795300 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.795982 (+ 682us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.796104 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260929 02:52:54.797441 15486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4cf016056b44e4e904558518c3d271e" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" is_pre_election: true
I20260929 02:52:54.797948 15486 raft_consensus.cc:2454] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 0.
I20260929 02:52:54.798156 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4cf016056b44e4e904558518c3d271e" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" is_pre_election: true
I20260929 02:52:54.798650 15779 raft_consensus.cc:2454] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 0.
I20260929 02:52:54.798657 15564 leader_election.cc:302] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 029c3d91cc13429ab849c1c0dba9397b, 1dd686a3ff2b4e2ab287add4507b6de3; no voters:
I20260929 02:52:54.799052 15866 raft_consensus.cc:2781] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:54.799201 15866 raft_consensus.cc:482] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:54.799302 15866 raft_consensus.cc:3037] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:54.801437 15866 raft_consensus.cc:504] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.802052 15866 leader_election.cc:288] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 election: Requested vote from peers 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913), 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839)
I20260929 02:52:54.802439 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4cf016056b44e4e904558518c3d271e" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d591eb2be1c4ffd88fafb45fd4db875"
I20260929 02:52:54.802462 15486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4cf016056b44e4e904558518c3d271e" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd686a3ff2b4e2ab287add4507b6de3"
I20260929 02:52:54.802629 15779 raft_consensus.cc:3037] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:54.802678 15486 raft_consensus.cc:3037] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:54.804873 15486 raft_consensus.cc:2454] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 1.
I20260929 02:52:54.804879 15779 raft_consensus.cc:2454] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 1.
I20260929 02:52:54.805238 15564 leader_election.cc:302] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 029c3d91cc13429ab849c1c0dba9397b, 1dd686a3ff2b4e2ab287add4507b6de3, 2d591eb2be1c4ffd88fafb45fd4db875; no voters:
I20260929 02:52:54.805548 15866 raft_consensus.cc:2781] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:54.805986 15866 raft_consensus.cc:686] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 1 LEADER]: Becoming Leader. State: Replica: 029c3d91cc13429ab849c1c0dba9397b, State: Running, Role: LEADER
I20260929 02:52:54.806496 15866 consensus_queue.cc:237] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } }
I20260929 02:52:54.811440 14557 catalog_manager.cc:5495] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b reported cstate change: term changed from 0 to 1, leader changed from <none> to 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2). New cstate: current_term: 1 leader_uuid: "029c3d91cc13429ab849c1c0dba9397b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } health_report { overall_health: HEALTHY } } }
I20260929 02:52:54.857055 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.852439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.8.57.2:42449)
0929 02:52:54.852621 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.852626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.852666 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:52:54.852764 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.852768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.852853 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.852972 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:54.853246 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.853950 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.855225 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.855626 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.855719 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.856130 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:54.856645 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:54.856748 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.856785 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.856849 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:52:54.864609 15486 raft_consensus.cc:1261] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Refusing update from remote peer 029c3d91cc13429ab849c1c0dba9397b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:54.864610 15779 raft_consensus.cc:1261] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Refusing update from remote peer 029c3d91cc13429ab849c1c0dba9397b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:54.865536 15866 consensus_queue.cc:1035] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:54.865836 15869 consensus_queue.cc:1035] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:54.879668 15874 mvcc.cc:205] Tried to move back new op lower bound from 7334503935436283904 to 7334503935211032576. Current Snapshot: MvccSnapshot[applied={T|T < 7334503935436283904}]
I20260929 02:52:54.905097 14557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:54.905328 14557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:54.911247 14557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91a711a321ce416d941d97b5da125e6a: Sending DeleteTablet for 3 replicas of tablet c4cf016056b44e4e904558518c3d271e
I20260929 02:52:54.913007 15759 tablet_service.cc:1514] Processing DeleteTablet for tablet c4cf016056b44e4e904558518c3d271e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:54 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:55344
I20260929 02:52:54.913226 15466 tablet_service.cc:1514] Processing DeleteTablet for tablet c4cf016056b44e4e904558518c3d271e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:54 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:57932
I20260929 02:52:54.914755 15618 tablet_service.cc:1514] Processing DeleteTablet for tablet c4cf016056b44e4e904558518c3d271e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:54 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:39306
I20260929 02:52:54.923650 15887 tablet_replica.cc:330] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: stopping tablet replica
I20260929 02:52:54.924409 15887 raft_consensus.cc:2229] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:54.925174 15884 tablet_replica.cc:330] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: stopping tablet replica
I20260929 02:52:54.925933 15887 raft_consensus.cc:2258] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:54.926632 15884 raft_consensus.cc:2229] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:54.927176 15884 raft_consensus.cc:2258] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:54.930981 15884 ts_tablet_manager.cc:1905] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:54.931438 15887 ts_tablet_manager.cc:1905] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:54.941883 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 14526
I20260929 02:52:54.940040 15883 tablet_replica.cc:330] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: stopping tablet replica
I20260929 02:52:54.942493 15883 raft_consensus.cc:2229] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:54.942997 14588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:54.943534 15883 raft_consensus.cc:2258] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:54.947103 15887 ts_tablet_manager.cc:1918] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:54.948700 15883 ts_tablet_manager.cc:1905] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:54.950451 15884 ts_tablet_manager.cc:1918] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:54.952088 15884 log.cc:1196] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/c4cf016056b44e4e904558518c3d271e
I20260929 02:52:54.952752 15884 ts_tablet_manager.cc:1939] T c4cf016056b44e4e904558518c3d271e P 1dd686a3ff2b4e2ab287add4507b6de3: Deleting consensus metadata
I20260929 02:52:54.952998 15887 log.cc:1196] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/c4cf016056b44e4e904558518c3d271e
I20260929 02:52:54.953657 15887 ts_tablet_manager.cc:1939] T c4cf016056b44e4e904558518c3d271e P 029c3d91cc13429ab849c1c0dba9397b: Deleting consensus metadata
I20260929 02:52:54.963723 15883 ts_tablet_manager.cc:1918] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:54.963989 15883 log.cc:1196] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/c4cf016056b44e4e904558518c3d271e
I20260929 02:52:54.964565 15883 ts_tablet_manager.cc:1939] T c4cf016056b44e4e904558518c3d271e P 2d591eb2be1c4ffd88fafb45fd4db875: Deleting consensus metadata
I20260929 02:52:54.973160 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.915205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.8.57.61:39775)
0929 02:52:54.940309 (+ 25104us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.940316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.946326 (+ 6010us) server_negotiation.cc:407] Connection header received
0929 02:52:54.947327 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.947334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.947443 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.947544 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.949900 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.951133 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.954963 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.955657 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.955794 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.966271 (+ 10477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.966279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.966284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.966328 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.970071 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.970929 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.970935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.970938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.971095 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.971483 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.971488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.971491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.971849 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.972101 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.972453 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.972664 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24955,"thread_start_us":24842,"threads_started":1}
I20260929 02:52:54.988902 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:54.941059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.8.57.60:46531)
0929 02:52:54.959371 (+ 18312us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:54.959378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:54.973045 (+ 13667us) server_negotiation.cc:407] Connection header received
0929 02:52:54.973193 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:54.973199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:54.973285 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:54.973389 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:54.975174 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.976378 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.978202 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:54.978781 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:54.978903 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:54.982612 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:54.982619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:54.982624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:54.982664 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:54.986087 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.986864 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.986870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.986874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.987023 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:54.987409 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:54.987414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:54.987418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:54.987757 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:54.988033 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:54.988324 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 02:52:54.988540 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18148,"thread_start_us":18319,"threads_started":1}
I20260929 02:52:55.098623 14542 catalog_manager.cc:4841] TS 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913): tablet c4cf016056b44e4e904558518c3d271e (table test-table [id=765d70a4e3b0427faf10b84627e0ac3f]) successfully deleted
I20260929 02:52:55.099221 14544 catalog_manager.cc:4841] TS 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839): tablet c4cf016056b44e4e904558518c3d271e (table test-table [id=765d70a4e3b0427faf10b84627e0ac3f]) successfully deleted
I20260929 02:52:55.099447 14544 catalog_manager.cc:4841] TS 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2:42449): tablet c4cf016056b44e4e904558518c3d271e (table test-table [id=765d70a4e3b0427faf10b84627e0ac3f]) successfully deleted
I20260929 02:52:55.099560 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 14526
I20260929 02:52:55.151178 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:55.124043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.8.57.60:46531)
0929 02:52:55.124255 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:55.124262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:55.133672 (+ 9410us) server_negotiation.cc:407] Connection header received
0929 02:52:55.133977 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:55.133985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:55.134084 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:55.134164 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:55.136980 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:55.138129 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:55.140156 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:55.140767 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:55.140871 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:55.144965 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:55.144974 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:55.144979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:55.145022 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:55.148359 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:55.149152 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:55.149158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:55.149162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:55.149307 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:55.149676 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:55.149679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:55.149681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:55.149936 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:55.150231 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:55.150472 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 02:52:55.150669 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:52:55.153764 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:55.124404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.8.57.61:39775)
0929 02:52:55.124579 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:55.124584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:55.138304 (+ 13720us) server_negotiation.cc:407] Connection header received
0929 02:52:55.139193 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:55.139199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:55.139314 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:55.139382 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:55.142038 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:55.143101 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:55.144942 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:55.145348 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:55.145456 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:55.147724 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:55.147729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:55.147733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:55.147765 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:55.150652 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:55.151570 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:55.151576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:55.151579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:55.151748 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:55.152342 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:55.152347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:55.152351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:55.152707 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:55.152942 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:55.153226 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 02:52:55.153459 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:52:55.892935 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:55.892283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:44631)
0929 02:52:55.892665 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:55.892777 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":186,"threads_started":1}
W20260929 02:52:55.893292 15684 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:34705 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:55.898747 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:55.898077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:60571)
0929 02:52:55.898492 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:55.898595 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":223,"threads_started":1}
W20260929 02:52:55.899091 15532 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:34705 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:55.902453 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:55.901796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:52545)
0929 02:52:55.902163 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:55.902286 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":170,"threads_started":1}
W20260929 02:52:55.902782 15825 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:34705 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.412070 15900 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91a711a321ce416d941d97b5da125e6a)
I20260929 02:52:56.412364 15900 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:56.412472 15901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91a711a321ce416d941d97b5da125e6a)
I20260929 02:52:56.412699 15901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:56.413765 15900 leader_election.cc:288] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705), 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61:39775)
I20260929 02:52:56.413898 15901 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705), d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60:46531)
I20260929 02:52:56.414973 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.413958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53134)
0929 02:52:56.414596 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.414758 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"threads_started":1}
W20260929 02:52:56.417771 15340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.419626 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.414921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53142)
0929 02:52:56.417007 (+ 2086us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.417202 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1962,"thread_start_us":1827,"threads_started":1}
W20260929 02:52:56.419942 15143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.423641 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.423223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53158)
0929 02:52:56.423395 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.423479 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 02:52:56.423892 15143 leader_election.cc:334] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.425318 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.413999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:39775 (local address 127.0.0.1:60558)
0929 02:52:56.415353 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.415373 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:56.415522 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:56.417139 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:56.417145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:56.417184 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:56.417514 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:56.417529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.419317 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.419325 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:56.421590 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:56.421606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.424543 (+ 2937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.424549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:56.424749 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:56.424790 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 02:52:56.424860 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 02:52:56.425004 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1197,"threads_started":1}
I20260929 02:52:56.426124 15904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.414226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.8.57.61:39775)
0929 02:52:56.416799 (+ 2573us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:56.416806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:56.416831 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:52:56.416904 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:56.416909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:56.416980 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:56.417698 (+ 718us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:56.417864 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.419140 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.421804 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.424319 (+ 2515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.425498 (+ 1179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:56.425580 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:56.425625 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 02:52:56.425744 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":942,"server-negotiator.queue_time_us":2434,"thread_start_us":2229,"threads_started":1}
I20260929 02:52:56.426808 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.414302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:46531 (local address 127.0.0.1:33724)
0929 02:52:56.416068 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.416086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:56.416257 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:56.418159 (+ 1902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:56.418164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:56.418191 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:56.418553 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:56.418564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.420689 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.420695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:56.423164 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:56.423177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.425976 (+ 2799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.425986 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:56.426174 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:56.426203 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:52:56.426259 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 02:52:56.426445 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1628,"mutex_wait_us":1,"thread_start_us":1462,"threads_started":1}
I20260929 02:52:56.427794 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.415285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33724 (local address 127.8.57.60:46531)
0929 02:52:56.417836 (+ 2551us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:56.417841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:56.417860 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:52:56.417917 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:56.417921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:56.417982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:56.418711 (+ 729us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:56.418896 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.420458 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.423957 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:56.425755 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:56.427063 (+ 1308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:56.427188 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:56.427246 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 02:52:56.427405 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2416,"thread_start_us":2262,"threads_started":1}
I20260929 02:52:56.428236 15167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2e0708665e414be9b968e2a1ad9c822a" is_pre_election: true
I20260929 02:52:56.428539 15167 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e7ea0a14ca4aed94f44bad47c5fcfd in term 1.
I20260929 02:52:56.428644 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.428093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53170)
0929 02:52:56.428255 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.428430 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:52:56.428555 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e0708665e414be9b968e2a1ad9c822a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" is_pre_election: true
W20260929 02:52:56.428833 15340 leader_election.cc:334] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.428896 15364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e0708665e414be9b968e2a1ad9c822a in term 1.
I20260929 02:52:56.429313 15341 leader_election.cc:302] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e0708665e414be9b968e2a1ad9c822a, d6e7ea0a14ca4aed94f44bad47c5fcfd; no voters: 91a711a321ce416d941d97b5da125e6a
I20260929 02:52:56.429438 15141 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e0708665e414be9b968e2a1ad9c822a, d6e7ea0a14ca4aed94f44bad47c5fcfd; no voters: 91a711a321ce416d941d97b5da125e6a
I20260929 02:52:56.429734 15900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 02:52:56.429736 15901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 02:52:56.429849 15901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91a711a321ce416d941d97b5da125e6a)
I20260929 02:52:56.429863 15900 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91a711a321ce416d941d97b5da125e6a)
I20260929 02:52:56.429932 15901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:52:56.429948 15900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:52:56.432883 15901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:56.432883 15900 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:56.433583 15901 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 2 election: Requested vote from peers 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705), d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60:46531)
I20260929 02:52:56.433612 15900 leader_election.cc:288] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [CANDIDATE]: Term 2 election: Requested vote from peers 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705), 2e0708665e414be9b968e2a1ad9c822a (127.8.57.61:39775)
I20260929 02:52:56.434185 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.433680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53180)
0929 02:52:56.433903 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.433993 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260929 02:52:56.434206 15167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2e0708665e414be9b968e2a1ad9c822a"
I20260929 02:52:56.434280 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e0708665e414be9b968e2a1ad9c822a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
I20260929 02:52:56.434532 15167 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d6e7ea0a14ca4aed94f44bad47c5fcfd in current term 2: Already voted for candidate 2e0708665e414be9b968e2a1ad9c822a in this term.
I20260929 02:52:56.434545 15364 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2e0708665e414be9b968e2a1ad9c822a in current term 2: Already voted for candidate d6e7ea0a14ca4aed94f44bad47c5fcfd in this term.
I20260929 02:52:56.434599 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.433671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53182)
0929 02:52:56.434385 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.434464 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546}
I20260929 02:52:56.435354 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.434955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53188)
0929 02:52:56.435132 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.435221 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 02:52:56.435598 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.435268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:53192)
0929 02:52:56.435393 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.435461 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 02:52:56.435556 15143 leader_election.cc:334] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.435724 15143 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e0708665e414be9b968e2a1ad9c822a; no voters: 91a711a321ce416d941d97b5da125e6a, d6e7ea0a14ca4aed94f44bad47c5fcfd
W20260929 02:52:56.435791 15340 leader_election.cc:334] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:56.436005 15901 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 02:52:56.435987 15340 leader_election.cc:302] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6e7ea0a14ca4aed94f44bad47c5fcfd; no voters: 2e0708665e414be9b968e2a1ad9c822a, 91a711a321ce416d941d97b5da125e6a
I20260929 02:52:56.436288 15900 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 02:52:56.894495 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.893844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:50639)
0929 02:52:56.894244 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.894326 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":208,"threads_started":1}
I20260929 02:52:56.900252 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.899647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:50181)
0929 02:52:56.900015 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.900090 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":173,"threads_started":1}
I20260929 02:52:56.904043 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:56.903372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:38639)
0929 02:52:56.903748 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:56.903823 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":202,"threads_started":1}
I20260929 02:52:57.895851 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:57.895210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:45713)
0929 02:52:57.895617 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:57.895699 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":205,"threads_started":1}
I20260929 02:52:57.901644 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:57.900959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:34805)
0929 02:52:57.901378 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:57.901462 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":233,"threads_started":1}
I20260929 02:52:57.905320 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:57.904661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:59705)
0929 02:52:57.905079 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:57.905172 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":228,"threads_started":1}
I20260929 02:52:58.240214 15916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:58.240388 15916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:58.240911 15916 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705), d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60:46531)
I20260929 02:52:58.241400 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e0708665e414be9b968e2a1ad9c822a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" is_pre_election: true
I20260929 02:52:58.241601 15364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e0708665e414be9b968e2a1ad9c822a in term 2.
I20260929 02:52:58.241905 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.241032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36446)
0929 02:52:58.241625 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.241710 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":359,"threads_started":1}
I20260929 02:52:58.242031 15141 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e0708665e414be9b968e2a1ad9c822a, d6e7ea0a14ca4aed94f44bad47c5fcfd; no voters:
I20260929 02:52:58.242280 15916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260929 02:52:58.242395 15916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:58.242494 15916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 2 FOLLOWER]: Advancing to term 3
I20260929 02:52:58.243176 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.242843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36458)
0929 02:52:58.242999 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.243068 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:52:58.243341 15143 leader_election.cc:334] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:58.244808 15916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:58.245258 15916 leader_election.cc:288] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 3 election: Requested vote from peers 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705), d6e7ea0a14ca4aed94f44bad47c5fcfd (127.8.57.60:46531)
I20260929 02:52:58.245693 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.245348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36472)
0929 02:52:58.245539 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.245593 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260929 02:52:58.245639 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e0708665e414be9b968e2a1ad9c822a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd"
I20260929 02:52:58.245806 15364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 2 FOLLOWER]: Advancing to term 3
I20260929 02:52:58.246551 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.246276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36488)
0929 02:52:58.246385 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.246439 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 02:52:58.246686 15143 leader_election.cc:334] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
I20260929 02:52:58.248047 15364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e0708665e414be9b968e2a1ad9c822a in term 3.
I20260929 02:52:58.248404 15141 leader_election.cc:302] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e0708665e414be9b968e2a1ad9c822a, d6e7ea0a14ca4aed94f44bad47c5fcfd; no voters: 91a711a321ce416d941d97b5da125e6a
I20260929 02:52:58.248678 15916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 3 FOLLOWER]: Leader election won for term 3
I20260929 02:52:58.249006 15916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [term 3 LEADER]: Becoming Leader. State: Replica: 2e0708665e414be9b968e2a1ad9c822a, State: Running, Role: LEADER
I20260929 02:52:58.249338 15916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } }
I20260929 02:52:58.251358 15919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e0708665e414be9b968e2a1ad9c822a. Latest consensus state: current_term: 3 leader_uuid: "2e0708665e414be9b968e2a1ad9c822a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:58.251583 15919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:58.252182 15921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:52:58.255417 15921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:52:58.256212 15921 catalog_manager.cc:1259] Loaded cluster ID: 7438bba307764cdc930a983764e22d2c
I20260929 02:52:58.256280 15921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:52:58.257359 15921 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:52:58.258144 15921 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Loaded TSK: 0
I20260929 02:52:58.258919 15921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:52:58.320874 15157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:52:58.328025 15157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:52:58.332137 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.331702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36500)
0929 02:52:58.331906 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.331989 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 02:52:58.332221 15364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [term 3 FOLLOWER]: Refusing update from remote peer 2e0708665e414be9b968e2a1ad9c822a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260929 02:52:58.332891 15919 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260929 02:52:58.333110 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.332707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36510)
0929 02:52:58.332898 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.332980 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 02:52:58.333325 15143 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a -> Peer 91a711a321ce416d941d97b5da125e6a (127.8.57.62:34705): Couldn't send request to peer 91a711a321ce416d941d97b5da125e6a. Status: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 02:52:58.335461 15923 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e0708665e414be9b968e2a1ad9c822a. Latest consensus state: current_term: 3 leader_uuid: "2e0708665e414be9b968e2a1ad9c822a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:58.335758 15923 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:58.337941 15923 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2e0708665e414be9b968e2a1ad9c822a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:58.338325 15923 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e7ea0a14ca4aed94f44bad47c5fcfd [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:52:58.344761 15928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2e0708665e414be9b968e2a1ad9c822a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "91a711a321ce416d941d97b5da125e6a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34705 } } peers { permanent_uuid: "2e0708665e414be9b968e2a1ad9c822a" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 39775 } attrs { promote: false } } peers { permanent_uuid: "d6e7ea0a14ca4aed94f44bad47c5fcfd" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 46531 } attrs { promote: false } } }
I20260929 02:52:58.345120 15928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a [sys.catalog]: This master's current role is: LEADER
I20260929 02:52:58.364753 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.352019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:34839 (local address 127.0.0.1:50160)
0929 02:52:58.352162 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.352182 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:58.352299 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:58.360099 (+ 7800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:58.360102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:58.360119 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:58.360333 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:58.360340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.361295 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.361300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:58.362666 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:58.362679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.364023 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.364030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:58.364204 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.364227 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:52:58.364303 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:52:58.364436 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 02:52:58.364771 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.352133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.8.57.1:34839)
0929 02:52:58.359153 (+ 7020us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.359160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.359181 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:52:58.359256 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.359262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.359346 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.360083 (+ 737us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:58.360474 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.361175 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.362811 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.363858 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.363993 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.364098 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.364345 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.364470 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6875,"thread_start_us":7073,"threads_started":1}
I20260929 02:52:58.370260 15466 tablet_service.cc:1467] Processing CreateTablet for tablet a9cef1e5391b42c5848ebb4321179543 (DEFAULT_TABLE table=test-table [id=d9490d5cfc044c9aa8b207ec512d01ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:58.370692 15466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cef1e5391b42c5848ebb4321179543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:58.374816 15933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.352934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.8.57.3:45913)
0929 02:52:58.368924 (+ 15990us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.368931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.368949 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:52:58.369014 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.369018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.369088 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.369188 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:58.369542 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.370379 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.372744 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.373878 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.374009 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.374109 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.374388 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.374519 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15858,"thread_start_us":15749,"threads_started":1}
I20260929 02:52:58.374857 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.352849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45913 (local address 127.0.0.1:54072)
0929 02:52:58.365276 (+ 12427us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.365291 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:58.365364 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:58.369194 (+ 3830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:58.369198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:58.369220 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:58.369369 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:58.369379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.370528 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.370534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:58.372573 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:58.372588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.374028 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.374034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:58.374236 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.374270 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:52:58.374360 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:52:58.374496 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12351,"mutex_wait_us":70,"thread_start_us":6605,"threads_started":1}
I20260929 02:52:58.375254 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.352424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:42449 (local address 127.0.0.1:56832)
0929 02:52:58.359698 (+ 7274us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.359712 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:52:58.359801 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:52:58.368558 (+ 8757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:52:58.368562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:52:58.368586 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:52:58.368995 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:58.369006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.370300 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.370306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:58.372564 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:52:58.372575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.374505 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.374511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:52:58.374695 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.374725 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:52:58.374801 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:52:58.374931 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7157,"mutex_wait_us":69,"thread_start_us":17013,"threads_started":1}
I20260929 02:52:58.375912 15931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.352806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.8.57.2:42449)
0929 02:52:58.368177 (+ 15371us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.368184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.368203 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:52:58.368271 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.368276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.368350 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.368441 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:52:58.369199 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.370153 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.372716 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.373712 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.373825 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.375503 (+ 1678us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.375553 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.375667 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15238,"thread_start_us":15139,"threads_started":1}
I20260929 02:52:58.375898 15759 tablet_service.cc:1467] Processing CreateTablet for tablet a9cef1e5391b42c5848ebb4321179543 (DEFAULT_TABLE table=test-table [id=d9490d5cfc044c9aa8b207ec512d01ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:58.376371 15759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cef1e5391b42c5848ebb4321179543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:58.376654 15618 tablet_service.cc:1467] Processing CreateTablet for tablet a9cef1e5391b42c5848ebb4321179543 (DEFAULT_TABLE table=test-table [id=d9490d5cfc044c9aa8b207ec512d01ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:52:58.376968 15618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cef1e5391b42c5848ebb4321179543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:52:58.380427 15934 tablet_bootstrap.cc:492] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Bootstrap starting.
I20260929 02:52:58.382189 15934 tablet_bootstrap.cc:654] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:58.382476 15935 tablet_bootstrap.cc:492] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Bootstrap starting.
I20260929 02:52:58.384356 15935 tablet_bootstrap.cc:654] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:58.384438 15936 tablet_bootstrap.cc:492] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Bootstrap starting.
I20260929 02:52:58.386183 15936 tablet_bootstrap.cc:654] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Neither blocks nor log segments found. Creating new log.
I20260929 02:52:58.386399 15935 tablet_bootstrap.cc:492] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: No bootstrap required, opened a new log
I20260929 02:52:58.386444 15934 tablet_bootstrap.cc:492] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: No bootstrap required, opened a new log
I20260929 02:52:58.386540 15935 ts_tablet_manager.cc:1397] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260929 02:52:58.386575 15934 ts_tablet_manager.cc:1397] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 02:52:58.387272 15935 raft_consensus.cc:348] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.387240 15934 raft_consensus.cc:348] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.387446 15935 raft_consensus.cc:374] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:58.387477 15934 raft_consensus.cc:374] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:58.387547 15935 raft_consensus.cc:729] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 029c3d91cc13429ab849c1c0dba9397b, State: Initialized, Role: FOLLOWER
I20260929 02:52:58.387593 15934 raft_consensus.cc:729] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd686a3ff2b4e2ab287add4507b6de3, State: Initialized, Role: FOLLOWER
I20260929 02:52:58.387774 15935 consensus_queue.cc:260] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.387861 15934 consensus_queue.cc:260] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.388208 15936 tablet_bootstrap.cc:492] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: No bootstrap required, opened a new log
I20260929 02:52:58.388391 15936 ts_tablet_manager.cc:1397] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260929 02:52:58.388605 15935 ts_tablet_manager.cc:1428] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260929 02:52:58.389545 15934 ts_tablet_manager.cc:1428] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 02:52:58.390276 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.389459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:53583)
0929 02:52:58.390047 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.390140 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474}
I20260929 02:52:58.391997 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.391527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:33473)
0929 02:52:58.391724 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.391823 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260929 02:52:58.392226 15936 raft_consensus.cc:348] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.392477 15685 heartbeater.cc:503] Master 127.8.57.61:39775 was elected leader, sending a full tablet report...
I20260929 02:52:58.392488 15936 raft_consensus.cc:374] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:52:58.392586 15936 raft_consensus.cc:729] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d591eb2be1c4ffd88fafb45fd4db875, State: Initialized, Role: FOLLOWER
I20260929 02:52:58.392786 15936 consensus_queue.cc:260] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.393409 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.393051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:58919)
0929 02:52:58.393189 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.393266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 02:52:58.394889 15936 ts_tablet_manager.cc:1428] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260929 02:52:58.395540 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.395175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:52443)
0929 02:52:58.395323 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.395413 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 02:52:58.396468 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.396097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:42295)
0929 02:52:58.396268 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.396346 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 02:52:58.396842 15826 heartbeater.cc:503] Master 127.8.57.61:39775 was elected leader, sending a full tablet report...
I20260929 02:52:58.609915 15940 raft_consensus.cc:482] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:52:58.610150 15940 raft_consensus.cc:504] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.610954 15940 leader_election.cc:288] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839), 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913)
I20260929 02:52:58.611514 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cef1e5391b42c5848ebb4321179543" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" is_pre_election: true
I20260929 02:52:58.611515 15486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cef1e5391b42c5848ebb4321179543" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" is_pre_election: true
I20260929 02:52:58.611771 15779 raft_consensus.cc:2454] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 0.
I20260929 02:52:58.611771 15486 raft_consensus.cc:2454] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 0.
I20260929 02:52:58.612241 15564 leader_election.cc:302] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 029c3d91cc13429ab849c1c0dba9397b, 1dd686a3ff2b4e2ab287add4507b6de3; no voters:
I20260929 02:52:58.612488 15940 raft_consensus.cc:2781] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:52:58.612607 15940 raft_consensus.cc:482] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:52:58.612685 15940 raft_consensus.cc:3037] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:58.614686 15940 raft_consensus.cc:504] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.615167 15940 leader_election.cc:288] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 election: Requested vote from peers 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839), 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913)
I20260929 02:52:58.615605 15486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cef1e5391b42c5848ebb4321179543" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd686a3ff2b4e2ab287add4507b6de3"
I20260929 02:52:58.615694 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cef1e5391b42c5848ebb4321179543" candidate_uuid: "029c3d91cc13429ab849c1c0dba9397b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d591eb2be1c4ffd88fafb45fd4db875"
I20260929 02:52:58.615787 15486 raft_consensus.cc:3037] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:58.615916 15779 raft_consensus.cc:3037] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:52:58.617936 15779 raft_consensus.cc:2454] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 1.
I20260929 02:52:58.617949 15486 raft_consensus.cc:2454] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 029c3d91cc13429ab849c1c0dba9397b in term 1.
I20260929 02:52:58.618283 15564 leader_election.cc:302] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 029c3d91cc13429ab849c1c0dba9397b, 1dd686a3ff2b4e2ab287add4507b6de3; no voters:
I20260929 02:52:58.618567 15940 raft_consensus.cc:2781] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:52:58.619020 15940 raft_consensus.cc:686] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 1 LEADER]: Becoming Leader. State: Replica: 029c3d91cc13429ab849c1c0dba9397b, State: Running, Role: LEADER
I20260929 02:52:58.619303 15940 consensus_queue.cc:237] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } }
I20260929 02:52:58.620244 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.619841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:52397)
0929 02:52:58.620008 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.620082 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:52:58.621908 15156 catalog_manager.cc:5495] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b reported cstate change: term changed from 0 to 1, leader changed from <none> to 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2). New cstate: current_term: 1 leader_uuid: "029c3d91cc13429ab849c1c0dba9397b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "029c3d91cc13429ab849c1c0dba9397b" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 } health_report { overall_health: UNKNOWN } } }
I20260929 02:52:58.653280 15931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.649018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.8.57.2:42449)
0929 02:52:58.649229 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.649235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.649255 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:52:58.649320 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.649324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.649384 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.649456 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:52:58.649802 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.650470 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.651707 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.652176 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.652285 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.652636 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:52:58.652735 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:52:58.652806 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.652943 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.653017 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:52:58.656147 15779 raft_consensus.cc:1261] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Refusing update from remote peer 029c3d91cc13429ab849c1c0dba9397b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:58.656180 15486 raft_consensus.cc:1261] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Refusing update from remote peer 029c3d91cc13429ab849c1c0dba9397b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:52:58.656734 15943 consensus_queue.cc:1035] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dd686a3ff2b4e2ab287add4507b6de3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:58.657029 15940 consensus_queue.cc:1035] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d591eb2be1c4ffd88fafb45fd4db875" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:52:58.659219 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.658747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:40381)
0929 02:52:58.658960 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.659051 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260929 02:52:58.659977 15533 heartbeater.cc:503] Master 127.8.57.61:39775 was elected leader, sending a full tablet report...
I20260929 02:52:58.660853 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.660424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:55005)
0929 02:52:58.660591 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.660692 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:52:58.663404 15946 mvcc.cc:205] Tried to move back new op lower bound from 7334503950970011648 to 7334503950826749952. Current Snapshot: MvccSnapshot[applied={T|T < 7334503950970011648}]
I20260929 02:52:58.664952 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.664335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.3:41537)
0929 02:52:58.664669 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.664761 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260929 02:52:58.668651 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.668250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.1:55339)
0929 02:52:58.668394 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.668488 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 02:52:58.672012 15947 mvcc.cc:205] Tried to move back new op lower bound from 7334503950970011648 to 7334503950826749952. Current Snapshot: MvccSnapshot[applied={T|T < 7334503950970011648}]
I20260929 02:52:58.674150 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.673699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:50759)
0929 02:52:58.673922 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.674014 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260929 02:52:58.677093 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.676641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.8.57.2:57153)
0929 02:52:58.676868 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.676955 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260929 02:52:58.678747 15157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:58.678946 15157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:52:58.683498 15157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e0708665e414be9b968e2a1ad9c822a: Sending DeleteTablet for 3 replicas of tablet a9cef1e5391b42c5848ebb4321179543
I20260929 02:52:58.684311 15466 tablet_service.cc:1514] Processing DeleteTablet for tablet a9cef1e5391b42c5848ebb4321179543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:58 UTC) from {username='oryx', principal='oryx/127.8.57.61@KRBTEST.COM'} at 127.0.0.1:50160
I20260929 02:52:58.684422 15618 tablet_service.cc:1514] Processing DeleteTablet for tablet a9cef1e5391b42c5848ebb4321179543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:58 UTC) from {username='oryx', principal='oryx/127.8.57.61@KRBTEST.COM'} at 127.0.0.1:56832
I20260929 02:52:58.685065 15759 tablet_service.cc:1514] Processing DeleteTablet for tablet a9cef1e5391b42c5848ebb4321179543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:52:58 UTC) from {username='oryx', principal='oryx/127.8.57.61@KRBTEST.COM'} at 127.0.0.1:54072
I20260929 02:52:58.685313 15955 tablet_replica.cc:330] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: stopping tablet replica
I20260929 02:52:58.685571 15955 raft_consensus.cc:2229] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:58.686224 15957 tablet_replica.cc:330] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: stopping tablet replica
I20260929 02:52:58.686651 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 15324
I20260929 02:52:58.686447 15957 raft_consensus.cc:2229] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:52:58.687282 15957 raft_consensus.cc:2258] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:58.687374 15955 raft_consensus.cc:2258] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:58.688987 15955 ts_tablet_manager.cc:1905] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:58.692960 15957 ts_tablet_manager.cc:1905] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:58.694777 15955 ts_tablet_manager.cc:1918] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:58.694980 15955 log.cc:1196] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/a9cef1e5391b42c5848ebb4321179543
I20260929 02:52:58.695504 15955 ts_tablet_manager.cc:1939] T a9cef1e5391b42c5848ebb4321179543 P 1dd686a3ff2b4e2ab287add4507b6de3: Deleting consensus metadata
I20260929 02:52:58.696710 15143 catalog_manager.cc:4841] TS 1dd686a3ff2b4e2ab287add4507b6de3 (127.8.57.1:34839): tablet a9cef1e5391b42c5848ebb4321179543 (table test-table [id=d9490d5cfc044c9aa8b207ec512d01ff]) successfully deleted
I20260929 02:52:58.699823 15957 ts_tablet_manager.cc:1918] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:58.699971 15957 log.cc:1196] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/a9cef1e5391b42c5848ebb4321179543
I20260929 02:52:58.700410 15957 ts_tablet_manager.cc:1939] T a9cef1e5391b42c5848ebb4321179543 P 2d591eb2be1c4ffd88fafb45fd4db875: Deleting consensus metadata
I20260929 02:52:58.701469 15141 catalog_manager.cc:4841] TS 2d591eb2be1c4ffd88fafb45fd4db875 (127.8.57.3:45913): tablet a9cef1e5391b42c5848ebb4321179543 (table test-table [id=d9490d5cfc044c9aa8b207ec512d01ff]) successfully deleted
I20260929 02:52:58.703863 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.688493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.8.57.1:34839)
0929 02:52:58.688682 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.688688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.688707 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:52:58.688763 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.688769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.688847 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.688945 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:58.690533 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.691717 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.693546 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.694212 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.694302 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.697478 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:58.697492 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:58.697507 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:58.697541 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:58.700358 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:58.701264 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:58.701275 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:58.701280 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:58.701445 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:58.701892 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:58.701897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:58.701901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:58.702394 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:58.702667 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.703255 (+ 588us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.703506 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:52:58.704275 15956 tablet_replica.cc:330] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: stopping tablet replica
I20260929 02:52:58.704569 15956 raft_consensus.cc:2229] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:52:58.704977 15956 raft_consensus.cc:2258] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:52:58.705066 15526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:58.705937 15956 ts_tablet_manager.cc:1905] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:52:58.712764 15956 ts_tablet_manager.cc:1918] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:52:58.712911 15956 log.cc:1196] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/a9cef1e5391b42c5848ebb4321179543
I20260929 02:52:58.713325 15956 ts_tablet_manager.cc:1939] T a9cef1e5391b42c5848ebb4321179543 P 029c3d91cc13429ab849c1c0dba9397b: Deleting consensus metadata
I20260929 02:52:58.714458 15143 catalog_manager.cc:4841] TS 029c3d91cc13429ab849c1c0dba9397b (127.8.57.2:42449): tablet a9cef1e5391b42c5848ebb4321179543 (table test-table [id=d9490d5cfc044c9aa8b207ec512d01ff]) successfully deleted
I20260929 02:52:58.796360 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 15324
I20260929 02:52:58.810079 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 15546
I20260929 02:52:58.822687 15931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.810747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.8.57.2:42449)
0929 02:52:58.810910 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.810916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.810992 (+ 76us) server_negotiation.cc:407] Connection header received
0929 02:52:58.811185 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.811190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.811262 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.811344 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:58.812669 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.813372 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.814537 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.814954 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.815063 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.817461 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:58.817475 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:58.817491 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:58.817523 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:58.820040 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:58.820637 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:58.820654 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:58.820656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:58.820789 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:58.821120 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:58.821125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:58.821128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:58.821526 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:58.821789 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.822073 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.822271 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:52:58.823320 15678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:58.832398 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.831923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36524)
0929 02:52:58.832099 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.832214 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 02:52:58.833748 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.833419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34705 (local address 127.0.0.1:36536)
0929 02:52:58.833552 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 02:52:58.833628 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 02:52:58.940604 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 15546
I20260929 02:52:58.957295 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 15696
I20260929 02:52:58.969385 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:52:58.958051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.8.57.3:45913)
0929 02:52:58.958459 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0929 02:52:58.958463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:52:58.958480 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:52:58.958548 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:52:58.958552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:52:58.958610 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:52:58.958696 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:52:58.959895 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.960627 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.961724 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:52:58.962101 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:52:58.962184 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:52:58.964551 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:52:58.964565 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:52:58.964580 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:52:58.964613 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:52:58.967213 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:58.967791 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:58.967797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:58.967799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:58.967958 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:52:58.968256 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:52:58.968259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:52:58.968261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:52:58.968583 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:52:58.968790 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:52:58.968974 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 02:52:58.969127 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":215,"threads_started":1}
I20260929 02:52:58.969969 15819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:59.060669 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 15696
I20260929 02:52:59.074061 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 15019
I20260929 02:52:59.075096 15187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:59.195912 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 15019
W20260929 02:52:59.212693 15340 connection.cc:535] server connection from 127.0.0.1:33724 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 02:52:59.213027 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 15120
I20260929 02:52:59.214102 15384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:52:59.326712 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 15120
2026-09-29T02:52:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11249 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:52:59 dist-test-slave-1dkt krb5kdc[15968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:52:59 dist-test-slave-1dkt krb5kdc[15968](info): set up 2 sockets
Sep 29 02:52:59 dist-test-slave-1dkt krb5kdc[15968](info): commencing operation
krb5kdc: starting...
W20260929 02:53:00.320784 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:53:00 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab.
Entry for principal oryx/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab.
I20260929 02:53:00.527827 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37087
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab
--principal=oryx/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:00.837123 15981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:00.837456 15981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:00.837587 15981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:00.844861 15981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:00.844995 15981 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:53:00.845065 15981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:00.845165 15981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:00.845244 15981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:00.845331 15981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:00.855198 15981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37087
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.62.keytab
--principal=oryx/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:00.856446 15981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:00.857659 15981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:00.869117 15987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:00.869207 15981 server_base.cc:1034] running on GCE node
W20260929 02:53:00.869310 15989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:00.869201 15986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:00.870215 15981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:00.871577 15981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:00.872769 15981 hybrid_clock.cc:648] HybridClock initialized: now 1790650380872687 us; error 63 us; skew 500 ppm
I20260929 02:53:00.873070 15981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:00 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650380, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:00.879840 15981 init.cc:377] Logged in from keytab as oryx/127.8.57.62@KRBTEST.COM (short username oryx)
I20260929 02:53:00.881376 15981 webserver.cc:460] Webserver started at http://127.8.57.62:34929/ using document root <none> and password file <none>
I20260929 02:53:00.882032 15981 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:00.882113 15981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:00.882390 15981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:00.885331 15981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "94869980840c4e8e83d9f5c5b2af3c44"
format_stamp: "Formatted at 2026-09-29 02:53:00 on dist-test-slave-1dkt"
server_key: "12e08cc0ed0e175f3a833e31133a712e"
server_key_iv: "cf81cb5fb420e8bcd603c6086d6c668b"
server_key_version: "encryptionkey@0"
I20260929 02:53:00.886085 15981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "94869980840c4e8e83d9f5c5b2af3c44"
format_stamp: "Formatted at 2026-09-29 02:53:00 on dist-test-slave-1dkt"
server_key: "12e08cc0ed0e175f3a833e31133a712e"
server_key_iv: "cf81cb5fb420e8bcd603c6086d6c668b"
server_key_version: "encryptionkey@0"
I20260929 02:53:00.891029 15981 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260929 02:53:00.894410 15996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:00.895709 15981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:53:00.895861 15981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "94869980840c4e8e83d9f5c5b2af3c44"
format_stamp: "Formatted at 2026-09-29 02:53:00 on dist-test-slave-1dkt"
server_key: "12e08cc0ed0e175f3a833e31133a712e"
server_key_iv: "cf81cb5fb420e8bcd603c6086d6c668b"
server_key_version: "encryptionkey@0"
I20260929 02:53:00.896086 15981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:00.915839 15981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:00.919902 15981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:00.920142 15981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:00.939474 15981 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:37087
I20260929 02:53:00.939492 16057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:37087 every 8 connection(s)
I20260929 02:53:00.940819 15981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:53:00.944448 16058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:00.946460 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 15981
I20260929 02:53:00.946627 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:53:00.947222 8420 external_mini_cluster.cc:1428] Setting key 38caa6eac7243d7510a9141b39105b04
I20260929 02:53:00.955740 16058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Bootstrap starting.
I20260929 02:53:00.959369 16058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:00.960651 16058 log.cc:824] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:53:00 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:00.963348 16058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: No bootstrap required, opened a new log
I20260929 02:53:00.968937 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:00.949156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56022 (local address 127.8.57.62:37087)
0929 02:53:00.949920 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:00.949931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:00.949971 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:53:00.951662 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:00.951683 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:00.952111 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:00.952444 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:00.954051 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:00.955275 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:00.956952 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:00.957628 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:00.957773 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:00.962167 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:00.962176 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:00.962191 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:00.962224 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:00.965091 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:00.965728 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:00.965735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:00.965741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:00.965906 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:00.966266 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:00.966269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:00.966272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:00.966765 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:00.967037 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:00.967495 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 02:53:00.967921 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":342,"threads_started":1}
I20260929 02:53:00.971058 16058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94869980840c4e8e83d9f5c5b2af3c44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37087 } }
I20260929 02:53:00.971396 16058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:00.971493 16058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94869980840c4e8e83d9f5c5b2af3c44, State: Initialized, Role: FOLLOWER
I20260929 02:53:00.972028 16058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94869980840c4e8e83d9f5c5b2af3c44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37087 } }
I20260929 02:53:00.972240 16058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:53:00.972349 16058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:53:00.972486 16058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:00.974539 16058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94869980840c4e8e83d9f5c5b2af3c44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37087 } }
I20260929 02:53:00.974990 16058 leader_election.cc:302] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94869980840c4e8e83d9f5c5b2af3c44; no voters:
I20260929 02:53:00.975528 16058 leader_election.cc:288] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:53:00.975664 16063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:00.976301 16063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [term 1 LEADER]: Becoming Leader. State: Replica: 94869980840c4e8e83d9f5c5b2af3c44, State: Running, Role: LEADER
I20260929 02:53:00.977216 16063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94869980840c4e8e83d9f5c5b2af3c44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37087 } }
I20260929 02:53:00.978019 16058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:53:00.979681 16065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94869980840c4e8e83d9f5c5b2af3c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94869980840c4e8e83d9f5c5b2af3c44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37087 } } }
I20260929 02:53:00.980365 16065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:00.980436 16064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94869980840c4e8e83d9f5c5b2af3c44. Latest consensus state: current_term: 1 leader_uuid: "94869980840c4e8e83d9f5c5b2af3c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94869980840c4e8e83d9f5c5b2af3c44" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37087 } } }
I20260929 02:53:00.980674 16064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:00.982270 16071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:53:00.987527 16071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:53:00.995631 16071 catalog_manager.cc:1347] Generated new cluster ID: aa56526f53f048649e02705e825caf48
I20260929 02:53:00.995764 16071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:53:01.003891 16071 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:53:01.005208 16071 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:53:01.013525 16071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Generated new TSK 0
I20260929 02:53:01.014395 16071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab.
Entry for principal oryx/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab.
I20260929 02:53:01.076213 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37087
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:01.396631 16084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:01.396907 16084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:01.397037 16084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:01.404253 16084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:01.404352 16084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:01.404464 16084 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:53:01.404559 16084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:53:01.414089 16084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:37087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:01.415318 16084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:01.416639 16084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:01.428849 16089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:01.428890 16090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:01.429016 16092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:01.429826 16084 server_base.cc:1034] running on GCE node
I20260929 02:53:01.430378 16084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:01.431286 16084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:01.432487 16084 hybrid_clock.cc:648] HybridClock initialized: now 1790650381432411 us; error 54 us; skew 500 ppm
I20260929 02:53:01.432781 16084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:01 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650381, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:01.435829 16084 init.cc:377] Logged in from keytab as oryx/127.8.57.1@KRBTEST.COM (short username oryx)
I20260929 02:53:01.437264 16084 webserver.cc:460] Webserver started at http://127.8.57.1:35435/ using document root <none> and password file <none>
I20260929 02:53:01.437939 16084 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:01.438036 16084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:01.438329 16084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:01.441256 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "7ec497747a1e41efaf027ab0424adf02"
format_stamp: "Formatted at 2026-09-29 02:53:01 on dist-test-slave-1dkt"
server_key: "d2938f41acabbf2f3891005d32c147c7"
server_key_iv: "1befca0a493a510523945dbe29d1f142"
server_key_version: "encryptionkey@0"
I20260929 02:53:01.442034 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "7ec497747a1e41efaf027ab0424adf02"
format_stamp: "Formatted at 2026-09-29 02:53:01 on dist-test-slave-1dkt"
server_key: "d2938f41acabbf2f3891005d32c147c7"
server_key_iv: "1befca0a493a510523945dbe29d1f142"
server_key_version: "encryptionkey@0"
I20260929 02:53:01.447189 16084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:53:01.450661 16099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:01.451972 16084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:01.452109 16084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "7ec497747a1e41efaf027ab0424adf02"
format_stamp: "Formatted at 2026-09-29 02:53:01 on dist-test-slave-1dkt"
server_key: "d2938f41acabbf2f3891005d32c147c7"
server_key_iv: "1befca0a493a510523945dbe29d1f142"
server_key_version: "encryptionkey@0"
I20260929 02:53:01.452308 16084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:01.467567 16084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:01.471737 16084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:01.472038 16084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:01.474694 16084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:01.474792 16084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:01.474924 16084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:01.475039 16084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 29 02:53:01 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650381, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:01.495630 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:01.475183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:43205 (local address 127.8.57.62:37087)
0929 02:53:01.475935 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:01.475941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:01.476956 (+ 1015us) server_negotiation.cc:407] Connection header received
0929 02:53:01.479081 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:01.479096 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:01.479188 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:01.479299 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:01.481387 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.482353 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.483963 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.484507 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.484617 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:01.488777 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:01.488784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:01.488790 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:01.488834 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:01.492196 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:01.493148 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:01.493153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:01.493155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:01.493268 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:01.493825 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:01.493830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:01.493833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:01.494187 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:01.494333 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:01.495038 (+ 705us) server_negotiation.cc:299] Negotiation successful
0929 02:53:01.495266 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":541,"threads_started":1}
I20260929 02:53:01.496649 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:01.475864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.8.57.1:43205)
0929 02:53:01.476804 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:01.476848 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:01.478701 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:01.479593 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:01.479602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:01.480301 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:01.481205 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:01.481234 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.482509 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.482522 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:01.483807 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:01.483819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.484746 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.484754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:01.484961 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:01.485753 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:01.485787 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:01.488544 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:01.492356 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:01.492367 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:01.492385 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:01.492985 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:01.493411 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:01.493415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:01.493419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:01.493695 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:01.494323 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:01.494330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:01.494687 (+ 357us) client_negotiation.cc:769] Sending connection context
0929 02:53:01.494915 (+ 228us) client_negotiation.cc:240] Negotiation successful
0929 02:53:01.495443 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":429,"threads_started":1}
I20260929 02:53:01.517328 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:01.502733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:48705 (local address 127.8.57.62:37087)
0929 02:53:01.502875 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:01.502879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:01.503512 (+ 633us) server_negotiation.cc:407] Connection header received
0929 02:53:01.503694 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:01.503699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:01.503763 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:01.503840 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:01.505116 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.506299 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.508271 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.508857 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.508931 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:01.511380 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:01.511387 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:01.511393 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:01.511436 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:01.514814 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:01.515447 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:01.515452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:01.515456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:01.515617 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:01.516107 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:01.516111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:01.516113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:01.516379 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:01.516499 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:01.516929 (+ 430us) server_negotiation.cc:299] Negotiation successful
0929 02:53:01.517104 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 02:53:01.517678 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:01.502824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.8.57.1:48705)
0929 02:53:01.503402 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:01.503422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:01.503580 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:01.503899 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:01.503904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:01.504511 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:01.504937 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:01.504954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.506461 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.506476 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:01.508072 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:01.508091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.509172 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.509176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:01.509350 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:01.510033 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:01.510058 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:01.511154 (+ 1096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:01.514956 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:01.514960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:01.514962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:01.515349 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:01.515784 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:01.515790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:01.515794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:01.515980 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:01.516483 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:01.516488 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:01.516703 (+ 215us) client_negotiation.cc:769] Sending connection context
0929 02:53:01.516845 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 02:53:01.517212 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":329,"threads_started":1}
I20260929 02:53:01.528010 16084 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:33555
I20260929 02:53:01.528062 16219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:33555 every 8 connection(s)
I20260929 02:53:01.529145 16084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:53:01.531248 16220 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37087
I20260929 02:53:01.531440 16220 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:01.532150 16220 heartbeater.cc:511] Master 127.8.57.62:37087 requested a full tablet report, sending...
I20260929 02:53:01.533936 16013 ts_manager.cc:194] Registered new tserver with Master: 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555)
I20260929 02:53:01.535360 16013 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:43205
I20260929 02:53:01.538098 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 16084
I20260929 02:53:01.538270 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:53:01.538904 8420 external_mini_cluster.cc:1428] Setting key f8b9a56b8681950512bb2a7718eb6ded
WARNING: no policy specified for oryx/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab.
Entry for principal oryx/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab.
I20260929 02:53:01.580896 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37087
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:01.889935 16224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:01.890218 16224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:01.890343 16224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:01.897428 16224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:01.897528 16224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:01.897651 16224 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:53:01.897751 16224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:53:01.907539 16224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:37087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:01.908869 16224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:01.910156 16224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:01.922219 16229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:01.922187 16230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:01.922142 16232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:01.923105 16224 server_base.cc:1034] running on GCE node
I20260929 02:53:01.923674 16224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:01.924557 16224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:01.925772 16224 hybrid_clock.cc:648] HybridClock initialized: now 1790650381925709 us; error 38 us; skew 500 ppm
I20260929 02:53:01.926095 16224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:01 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650381, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:01.929036 16224 init.cc:377] Logged in from keytab as oryx/127.8.57.2@KRBTEST.COM (short username oryx)
I20260929 02:53:01.930373 16224 webserver.cc:460] Webserver started at http://127.8.57.2:39885/ using document root <none> and password file <none>
I20260929 02:53:01.930997 16224 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:01.931098 16224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:01.931389 16224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:01.934206 16224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "fcce2a22b2c74854b42555a803cf08bf"
format_stamp: "Formatted at 2026-09-29 02:53:01 on dist-test-slave-1dkt"
server_key: "f6e1146d9c0827af9f7432b2f03c58c9"
server_key_iv: "f08c5bffa1345ec249a9fdaf1f3c5518"
server_key_version: "encryptionkey@0"
I20260929 02:53:01.934926 16224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "fcce2a22b2c74854b42555a803cf08bf"
format_stamp: "Formatted at 2026-09-29 02:53:01 on dist-test-slave-1dkt"
server_key: "f6e1146d9c0827af9f7432b2f03c58c9"
server_key_iv: "f08c5bffa1345ec249a9fdaf1f3c5518"
server_key_version: "encryptionkey@0"
I20260929 02:53:01.939896 16224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:53:01.943987 16239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:01.945452 16224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:53:01.945600 16224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "fcce2a22b2c74854b42555a803cf08bf"
format_stamp: "Formatted at 2026-09-29 02:53:01 on dist-test-slave-1dkt"
server_key: "f6e1146d9c0827af9f7432b2f03c58c9"
server_key_iv: "f08c5bffa1345ec249a9fdaf1f3c5518"
server_key_version: "encryptionkey@0"
I20260929 02:53:01.945809 16224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:01.967965 16224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:01.972079 16224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:01.972297 16224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:01.974879 16224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:01.974967 16224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:01.975093 16224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:01.975210 16224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Sep 29 02:53:01 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650381, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:01.997849 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:01.975464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54513 (local address 127.8.57.62:37087)
0929 02:53:01.975650 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:01.975656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:01.977011 (+ 1355us) server_negotiation.cc:407] Connection header received
0929 02:53:01.979696 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:01.979702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:01.979790 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:01.979903 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:01.982328 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.983306 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.985500 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.986145 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.986276 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:01.990471 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:01.990478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:01.990485 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:01.990529 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:01.993882 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:01.994937 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:01.994941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:01.994944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:01.995138 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:01.995733 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:01.995738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:01.995741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:01.996144 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:01.996302 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:01.997252 (+ 950us) server_negotiation.cc:299] Negotiation successful
0929 02:53:01.997500 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:53:01.999547 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:01.975966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.8.57.2:54513)
0929 02:53:01.976825 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:01.976884 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:01.979368 (+ 2484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:01.980210 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:01.980223 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:01.981070 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:01.982113 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:01.982169 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.983460 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.983465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:01.985330 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:01.985347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:01.986416 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:01.986424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:01.986724 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:01.987524 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:01.987546 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:01.990227 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:01.994034 (+ 3807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:01.994041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:01.994054 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:01.994661 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:01.995272 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:01.995278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:01.995282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:01.995590 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:01.996283 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:01.996295 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:01.996778 (+ 483us) client_negotiation.cc:769] Sending connection context
0929 02:53:01.997093 (+ 315us) client_negotiation.cc:240] Negotiation successful
0929 02:53:01.997694 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":337,"threads_started":1}
I20260929 02:53:02.018904 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.005094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.8.57.2:59695)
0929 02:53:02.005698 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.005717 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.005834 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.006168 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.006171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.006591 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:02.006869 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.006878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.008395 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.008399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.009430 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.009443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.010395 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.010400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.010557 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.011148 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:02.011183 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:02.012177 (+ 994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:02.015820 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:02.015825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:02.015830 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:02.016495 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:02.016877 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:02.016886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:02.016890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:02.017063 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:02.017768 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:02.017773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:02.018009 (+ 236us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.018195 (+ 186us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.018558 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":327,"threads_started":1}
I20260929 02:53:02.018919 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.004928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:59695 (local address 127.8.57.62:37087)
0929 02:53:02.005116 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.005123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.005797 (+ 674us) server_negotiation.cc:407] Connection header received
0929 02:53:02.005933 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.005939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.006033 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.006150 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:02.006991 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.008250 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.009573 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.010225 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.010345 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.012434 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:02.012439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:02.012444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:02.012483 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:02.015674 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.016625 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.016630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.016634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.016772 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.017182 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.017187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.017190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.017555 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:02.017697 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.018314 (+ 617us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.018563 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:53:02.031298 16224 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:38541
I20260929 02:53:02.031319 16358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:38541 every 8 connection(s)
I20260929 02:53:02.032496 16224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:53:02.034502 16359 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37087
I20260929 02:53:02.034706 16359 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:02.035312 16359 heartbeater.cc:511] Master 127.8.57.62:37087 requested a full tablet report, sending...
I20260929 02:53:02.036571 16013 ts_manager.cc:194] Registered new tserver with Master: fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541)
I20260929 02:53:02.037364 16013 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:54513
I20260929 02:53:02.039829 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 16224
I20260929 02:53:02.040014 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:53:02.040470 8420 external_mini_cluster.cc:1428] Setting key dccb3e47b6220d85b55e1898da1672e3
WARNING: no policy specified for oryx/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab.
Entry for principal oryx/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab.
I20260929 02:53:02.082737 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37087
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:02.399232 16363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:02.399534 16363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:02.399667 16363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:02.407043 16363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:02.407163 16363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:02.407246 16363 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:53:02.407328 16363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:53:02.417164 16363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.57.62:37087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:02.418417 16363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:02.419718 16363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:02.432629 16368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:02.432663 16369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:02.432703 16371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:02.433354 16363 server_base.cc:1034] running on GCE node
I20260929 02:53:02.434003 16363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:02.434955 16363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:02.436175 16363 hybrid_clock.cc:648] HybridClock initialized: now 1790650382436093 us; error 57 us; skew 500 ppm
I20260929 02:53:02.436460 16363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:02 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:02.439675 16363 init.cc:377] Logged in from keytab as oryx/127.8.57.3@KRBTEST.COM (short username oryx)
I20260929 02:53:02.441231 16363 webserver.cc:460] Webserver started at http://127.8.57.3:40499/ using document root <none> and password file <none>
I20260929 02:53:02.441933 16363 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:02.442034 16363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:02.442334 16363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:02.445412 16363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "fd980ab14f9b475a9345497ced4ed82e"
format_stamp: "Formatted at 2026-09-29 02:53:02 on dist-test-slave-1dkt"
server_key: "0b061563e1e7933577a97c2ba477e122"
server_key_iv: "4f9bdd920f7ae26d05b67daeaf28c79b"
server_key_version: "encryptionkey@0"
I20260929 02:53:02.446213 16363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "fd980ab14f9b475a9345497ced4ed82e"
format_stamp: "Formatted at 2026-09-29 02:53:02 on dist-test-slave-1dkt"
server_key: "0b061563e1e7933577a97c2ba477e122"
server_key_iv: "4f9bdd920f7ae26d05b67daeaf28c79b"
server_key_version: "encryptionkey@0"
I20260929 02:53:02.451251 16363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:53:02.454931 16378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:02.456419 16363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:02.456588 16363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "fd980ab14f9b475a9345497ced4ed82e"
format_stamp: "Formatted at 2026-09-29 02:53:02 on dist-test-slave-1dkt"
server_key: "0b061563e1e7933577a97c2ba477e122"
server_key_iv: "4f9bdd920f7ae26d05b67daeaf28c79b"
server_key_version: "encryptionkey@0"
I20260929 02:53:02.456799 16363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:02.480700 16363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:02.485143 16363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:02.485452 16363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:02.488399 16363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:02.488480 16363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:02.488595 16363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:02.488648 16363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:53:02 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650382, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:02.514241 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.489080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:37983 (local address 127.8.57.62:37087)
0929 02:53:02.489263 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.489269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.490687 (+ 1418us) server_negotiation.cc:407] Connection header received
0929 02:53:02.493404 (+ 2717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.493409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.493485 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.493578 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:02.496052 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.497001 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.500059 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.500681 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.500808 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.506407 (+ 5599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:02.506415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:02.506421 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:02.506473 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:02.509691 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.510918 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.510926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.510929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.511119 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.511844 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.511850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.511853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.512264 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:02.512436 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.513589 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.513865 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:53:02.516083 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.489612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.8.57.3:37983)
0929 02:53:02.490478 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.490541 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.493057 (+ 2516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.493864 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.493879 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.494787 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:02.495790 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.495823 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.497185 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.497191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.499861 (+ 2670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.499898 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.500947 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.500957 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.501239 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.502112 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:02.502151 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:02.506056 (+ 3905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:02.509894 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:02.509907 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:02.509931 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:02.510756 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:02.511288 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:02.511295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:02.511299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:02.511651 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:02.512412 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:02.512424 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:02.513028 (+ 604us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.513422 (+ 394us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.514145 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":340,"threads_started":1}
I20260929 02:53:02.525395 16363 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:33127
I20260929 02:53:02.525483 16497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:33127 every 8 connection(s)
I20260929 02:53:02.526752 16363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:53:02.532131 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 16363
I20260929 02:53:02.532317 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:53:02.532905 8420 external_mini_cluster.cc:1428] Setting key 212c3f49cbcdb91f5d8356018e5dcb08
I20260929 02:53:02.529385 16498 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37087
I20260929 02:53:02.533883 16498 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:02.534848 16498 heartbeater.cc:511] Master 127.8.57.62:37087 requested a full tablet report, sending...
I20260929 02:53:02.536510 16013 ts_manager.cc:194] Registered new tserver with Master: fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:02.537408 16013 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:37983
I20260929 02:53:02.538038 16220 heartbeater.cc:503] Master 127.8.57.62:37087 was elected leader, sending a full tablet report...
I20260929 02:53:02.548134 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.523749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.8.57.3:37177)
0929 02:53:02.536370 (+ 12621us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.536388 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.536505 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.536855 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.536859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.537284 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:02.537531 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.537543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.538655 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.538659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.540056 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.540070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.540861 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.540866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.540992 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.541613 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:02.541631 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:02.542398 (+ 767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:02.545499 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:02.545503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:02.545506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:02.545937 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:02.546369 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:02.546373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:02.546376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:02.546522 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:02.547101 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:02.547106 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:02.547316 (+ 210us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.547432 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.547716 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12464,"thread_start_us":12316,"threads_started":1}
I20260929 02:53:02.548148 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.523763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:37177 (local address 127.8.57.62:37087)
0929 02:53:02.524011 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.524019 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.536474 (+ 12455us) server_negotiation.cc:407] Connection header received
0929 02:53:02.536594 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.536599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.536693 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.536793 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:02.537672 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.538488 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.540191 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.540665 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.540764 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.542640 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:02.542645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:02.542649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:02.542684 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:02.545363 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.546115 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.546119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.546122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.546256 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.546625 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.546630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.546633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.546952 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:02.547106 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.547572 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.547792 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:53:02.548702 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:53:02 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650382, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:53:02 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:02 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:53:02.596167 16108 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:02.585587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.8.57.62:37087)
0929 02:53:02.585752 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.585757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.586066 (+ 309us) server_negotiation.cc:407] Connection header received
0929 02:53:02.586219 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.586223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.586301 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.586381 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:02.587737 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.588559 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.589831 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.590394 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.590502 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.595864 (+ 5362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56024: BlockingRecv error: recv got EOF from 127.0.0.1:56024 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:53:02 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650382, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:02.613572 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.599143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.8.57.62:37087)
0929 02:53:02.599295 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.599299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.599614 (+ 315us) server_negotiation.cc:407] Connection header received
0929 02:53:02.599770 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.599774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.599895 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.599995 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:02.601314 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.602120 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.603417 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.603864 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.604011 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.608186 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:02.608192 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:02.608196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:02.608238 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:02.610852 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.611653 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.611657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.611660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.611795 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:02.612275 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:02.612278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:02.612280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:02.612569 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:02.612774 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.613115 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.613333 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:53:02.617636 16013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:53:02.624361 16013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:53:02.647497 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.637787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.8.57.1:33555)
0929 02:53:02.639034 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.639044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.639075 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:53:02.639181 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.639189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.639501 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.639729 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.640490 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.641945 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.644796 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.646342 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.646477 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.646702 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.647014 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.647202 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":934,"threads_started":1}
I20260929 02:53:02.647513 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.637182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:33555 (local address 127.0.0.1:59734)
0929 02:53:02.638296 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.638315 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.638509 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.639756 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.639765 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.639818 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.640264 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.640277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.642143 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.642148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.644607 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.644621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.646566 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.646572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.646762 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.646818 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.646957 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.647156 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":815,"threads_started":1}
I20260929 02:53:02.649466 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.636745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.8.57.2:38541)
0929 02:53:02.642470 (+ 5725us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.642477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.642498 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:53:02.642568 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.642573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.642749 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.642906 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.643503 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.644775 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.646727 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.648443 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.648670 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.648843 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.649022 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.649191 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5568,"thread_start_us":5418,"threads_started":1}
I20260929 02:53:02.649492 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.636559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:38541 (local address 127.0.0.1:51180)
0929 02:53:02.637341 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.637389 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.637583 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.642922 (+ 5339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.642927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.642951 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.643331 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.643341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.644925 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.644930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.646551 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.646564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.648670 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.648677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.648867 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.648898 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.648991 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.649134 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":109,"thread_start_us":328,"threads_started":1}
I20260929 02:53:02.650570 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.637688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33127 (local address 127.0.0.1:46814)
0929 02:53:02.640896 (+ 3208us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.640913 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.641043 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.641806 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.641811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.641836 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.642236 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.642251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.644317 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.644322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.646541 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.646557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.649778 (+ 3221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.649789 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.649952 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.649986 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.650078 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.650225 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3087,"mutex_wait_us":3,"thread_start_us":2960,"threads_started":1}
I20260929 02:53:02.650636 16521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.638220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.8.57.3:33127)
0929 02:53:02.640476 (+ 2256us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.640485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.641173 (+ 688us) server_negotiation.cc:407] Connection header received
0929 02:53:02.641327 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.641335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.641592 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.641792 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.642474 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.644152 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.646730 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.648136 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.648255 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.648497 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.650154 (+ 1657us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.650322 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2079,"thread_start_us":1866,"threads_started":1}
I20260929 02:53:02.652468 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 31c7d4e27a0047e8adeab5d141a55cfa (DEFAULT_TABLE table=test-table [id=7012a888e670431dadfa395752fdfbd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:02.652735 16289 tablet_service.cc:1467] Processing CreateTablet for tablet 31c7d4e27a0047e8adeab5d141a55cfa (DEFAULT_TABLE table=test-table [id=7012a888e670431dadfa395752fdfbd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:02.653863 16289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c7d4e27a0047e8adeab5d141a55cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:02.653931 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c7d4e27a0047e8adeab5d141a55cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:02.654336 16428 tablet_service.cc:1467] Processing CreateTablet for tablet 31c7d4e27a0047e8adeab5d141a55cfa (DEFAULT_TABLE table=test-table [id=7012a888e670431dadfa395752fdfbd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:02.655738 16428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c7d4e27a0047e8adeab5d141a55cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:02.666213 16522 tablet_bootstrap.cc:492] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Bootstrap starting.
I20260929 02:53:02.669484 16522 tablet_bootstrap.cc:654] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:02.669713 16523 tablet_bootstrap.cc:492] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Bootstrap starting.
I20260929 02:53:02.670516 16524 tablet_bootstrap.cc:492] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Bootstrap starting.
I20260929 02:53:02.670905 16522 log.cc:824] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:02.673923 16523 tablet_bootstrap.cc:654] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:02.674633 16524 tablet_bootstrap.cc:654] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:02.675319 16523 log.cc:824] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:02.676131 16524 log.cc:824] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:02.678287 16522 tablet_bootstrap.cc:492] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: No bootstrap required, opened a new log
I20260929 02:53:02.678360 16523 tablet_bootstrap.cc:492] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: No bootstrap required, opened a new log
I20260929 02:53:02.678606 16522 ts_tablet_manager.cc:1397] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:53:02.678683 16523 ts_tablet_manager.cc:1397] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:53:02.678869 16524 tablet_bootstrap.cc:492] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: No bootstrap required, opened a new log
I20260929 02:53:02.679113 16524 ts_tablet_manager.cc:1397] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260929 02:53:02.686453 16522 raft_consensus.cc:348] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.686851 16522 raft_consensus.cc:374] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:02.686947 16522 raft_consensus.cc:729] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcce2a22b2c74854b42555a803cf08bf, State: Initialized, Role: FOLLOWER
I20260929 02:53:02.687548 16522 consensus_queue.cc:260] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.688673 16359 heartbeater.cc:503] Master 127.8.57.62:37087 was elected leader, sending a full tablet report...
I20260929 02:53:02.689428 16522 ts_tablet_manager.cc:1428] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260929 02:53:02.689294 16524 raft_consensus.cc:348] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.689293 16523 raft_consensus.cc:348] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.689841 16523 raft_consensus.cc:374] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:02.689841 16524 raft_consensus.cc:374] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:02.689955 16524 raft_consensus.cc:729] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd980ab14f9b475a9345497ced4ed82e, State: Initialized, Role: FOLLOWER
I20260929 02:53:02.689955 16523 raft_consensus.cc:729] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Initialized, Role: FOLLOWER
I20260929 02:53:02.690598 16524 consensus_queue.cc:260] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.690598 16523 consensus_queue.cc:260] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.692538 16523 ts_tablet_manager.cc:1428] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 02:53:02.692704 16524 ts_tablet_manager.cc:1428] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260929 02:53:02.692962 16498 heartbeater.cc:503] Master 127.8.57.62:37087 was elected leader, sending a full tablet report...
I20260929 02:53:02.696172 16530 raft_consensus.cc:482] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:53:02.696470 16530 raft_consensus.cc:504] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.698195 16530 leader_election.cc:288] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:02.704440 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.698576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:37981 (local address 127.8.57.2:38541)
0929 02:53:02.698745 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.698749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.699333 (+ 584us) server_negotiation.cc:407] Connection header received
0929 02:53:02.699411 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.699416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.699490 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.699587 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.700111 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.700981 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.702771 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.703514 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.703640 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.703774 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.704061 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.704190 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:53:02.704512 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.698370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:38541 (local address 127.8.57.1:37981)
0929 02:53:02.698925 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.698945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.699070 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.699600 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.699605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.699638 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.699982 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.699992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.701099 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.701105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.702629 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.702641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.703711 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.703716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.703912 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.703949 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.704033 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.704175 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":271,"threads_started":1}
I20260929 02:53:02.704965 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.699040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33127 (local address 127.8.57.1:57559)
0929 02:53:02.699642 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.699659 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.699818 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.700226 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.700230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.700253 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.700565 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.700573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.701655 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.701659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.703115 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.703127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.704287 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.704293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.704424 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.704446 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.704529 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.704665 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":235,"threads_started":1}
I20260929 02:53:02.705644 16521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.699240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:57559 (local address 127.8.57.3:33127)
0929 02:53:02.699415 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.699421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.699730 (+ 309us) server_negotiation.cc:407] Connection header received
0929 02:53:02.700016 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.700021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.700124 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.700197 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.700719 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.701521 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.703253 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.704087 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.704197 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.704297 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.705285 (+ 988us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.705415 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:53:02.705760 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c7d4e27a0047e8adeab5d141a55cfa" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf" is_pre_election: true
I20260929 02:53:02.706315 16314 raft_consensus.cc:2454] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:02.706564 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c7d4e27a0047e8adeab5d141a55cfa" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e" is_pre_election: true
I20260929 02:53:02.706941 16103 leader_election.cc:302] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fcce2a22b2c74854b42555a803cf08bf; no voters:
I20260929 02:53:02.707093 16448 raft_consensus.cc:2454] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:02.707316 16530 raft_consensus.cc:2781] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:53:02.707451 16530 raft_consensus.cc:482] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:53:02.707551 16530 raft_consensus.cc:3037] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:02.709892 16530 raft_consensus.cc:504] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.710481 16530 leader_election.cc:288] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Requested vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:02.711014 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c7d4e27a0047e8adeab5d141a55cfa" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf"
I20260929 02:53:02.711053 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c7d4e27a0047e8adeab5d141a55cfa" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e"
I20260929 02:53:02.711215 16314 raft_consensus.cc:3037] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:02.711225 16448 raft_consensus.cc:3037] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:02.713510 16314 raft_consensus.cc:2454] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:02.713505 16448 raft_consensus.cc:2454] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:02.713919 16103 leader_election.cc:302] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fd980ab14f9b475a9345497ced4ed82e; no voters:
I20260929 02:53:02.714236 16530 raft_consensus.cc:2781] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:02.714697 16530 raft_consensus.cc:686] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Running, Role: LEADER
I20260929 02:53:02.715229 16530 consensus_queue.cc:237] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } }
I20260929 02:53:02.720010 16011 catalog_manager.cc:5495] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1). New cstate: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } health_report { overall_health: HEALTHY } } }
I20260929 02:53:02.735618 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.730573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.8.57.1:33555)
0929 02:53:02.730789 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.730795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.730813 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:53:02.730912 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.730915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.731013 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.731149 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:02.731502 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.732358 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.733628 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.734024 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.734129 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.734533 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:02.735056 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:02.735182 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.735282 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.735359 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:53:02.743781 16314 raft_consensus.cc:1261] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:02.743781 16448 raft_consensus.cc:1261] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:02.744747 16530 consensus_queue.cc:1035] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:02.745049 16533 consensus_queue.cc:1035] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:02.769129 16011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:02.769405 16011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:02.772845 16011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Sending DeleteTablet for 3 replicas of tablet 31c7d4e27a0047e8adeab5d141a55cfa
I20260929 02:53:02.774134 16289 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c7d4e27a0047e8adeab5d141a55cfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:02 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:51180
I20260929 02:53:02.774580 16428 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c7d4e27a0047e8adeab5d141a55cfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:02 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:46814
I20260929 02:53:02.774883 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c7d4e27a0047e8adeab5d141a55cfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:02 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:59734
I20260929 02:53:02.775228 16545 tablet_replica.cc:330] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: stopping tablet replica
I20260929 02:53:02.776825 16545 raft_consensus.cc:2229] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:02.776909 16011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:53:02.777587 16545 raft_consensus.cc:2258] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 02:53:02.777673 16011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:53:02.779803 16545 ts_tablet_manager.cc:1905] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:02.780308 16547 tablet_replica.cc:330] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: stopping tablet replica
I20260929 02:53:02.780745 16547 raft_consensus.cc:2229] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:53:02.781354 16547 raft_consensus.cc:2258] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:02.785471 16547 ts_tablet_manager.cc:1905] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:02.787099 16545 ts_tablet_manager.cc:1918] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:02.787309 16545 log.cc:1196] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/31c7d4e27a0047e8adeab5d141a55cfa
I20260929 02:53:02.787863 16545 ts_tablet_manager.cc:1939] T 31c7d4e27a0047e8adeab5d141a55cfa P fcce2a22b2c74854b42555a803cf08bf: Deleting consensus metadata
I20260929 02:53:02.789336 16427 tablet_service.cc:1467] Processing CreateTablet for tablet 001a67a195074cf0bcda889f472a5e5f (DEFAULT_TABLE table=test-table [id=0ef643a8e778491ab1a4fee54a4bcf3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:02.789561 16289 tablet_service.cc:1467] Processing CreateTablet for tablet 001a67a195074cf0bcda889f472a5e5f (DEFAULT_TABLE table=test-table [id=0ef643a8e778491ab1a4fee54a4bcf3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:02.789892 16427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001a67a195074cf0bcda889f472a5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:02.789834 16000 catalog_manager.cc:4841] TS fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541): tablet 31c7d4e27a0047e8adeab5d141a55cfa (table test-table [id=7012a888e670431dadfa395752fdfbd3]) successfully deleted
I20260929 02:53:02.789929 16289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001a67a195074cf0bcda889f472a5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:02.791443 16547 ts_tablet_manager.cc:1918] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:02.791592 16547 log.cc:1196] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/31c7d4e27a0047e8adeab5d141a55cfa
I20260929 02:53:02.792059 16547 ts_tablet_manager.cc:1939] T 31c7d4e27a0047e8adeab5d141a55cfa P 7ec497747a1e41efaf027ab0424adf02: Deleting consensus metadata
I20260929 02:53:02.793799 15999 catalog_manager.cc:4841] TS 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555): tablet 31c7d4e27a0047e8adeab5d141a55cfa (table test-table [id=7012a888e670431dadfa395752fdfbd3]) successfully deleted
I20260929 02:53:02.794505 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 001a67a195074cf0bcda889f472a5e5f (DEFAULT_TABLE table=test-table [id=0ef643a8e778491ab1a4fee54a4bcf3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:02.794957 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001a67a195074cf0bcda889f472a5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:02.795992 16524 tablet_bootstrap.cc:492] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Bootstrap starting.
I20260929 02:53:02.797593 16524 tablet_bootstrap.cc:654] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:02.806473 16522 tablet_bootstrap.cc:492] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Bootstrap starting.
I20260929 02:53:02.808190 16522 tablet_bootstrap.cc:654] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:02.815157 16523 tablet_bootstrap.cc:492] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Bootstrap starting.
I20260929 02:53:02.815629 16546 tablet_replica.cc:330] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: stopping tablet replica
I20260929 02:53:02.815953 16524 tablet_bootstrap.cc:492] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: No bootstrap required, opened a new log
I20260929 02:53:02.816123 16524 ts_tablet_manager.cc:1397] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.000s
I20260929 02:53:02.816146 16546 raft_consensus.cc:2229] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:02.816671 16546 raft_consensus.cc:2258] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:02.816938 16523 tablet_bootstrap.cc:654] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:02.817299 16524 raft_consensus.cc:348] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.817545 16524 raft_consensus.cc:374] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:02.817644 16524 raft_consensus.cc:729] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd980ab14f9b475a9345497ced4ed82e, State: Initialized, Role: FOLLOWER
I20260929 02:53:02.818827 16524 consensus_queue.cc:260] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.820387 16524 ts_tablet_manager.cc:1428] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 02:53:02.822047 16546 ts_tablet_manager.cc:1905] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:02.823770 16522 tablet_bootstrap.cc:492] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: No bootstrap required, opened a new log
I20260929 02:53:02.823958 16522 ts_tablet_manager.cc:1397] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.000s
I20260929 02:53:02.824769 16522 raft_consensus.cc:348] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.825001 16522 raft_consensus.cc:374] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:02.825089 16522 raft_consensus.cc:729] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcce2a22b2c74854b42555a803cf08bf, State: Initialized, Role: FOLLOWER
I20260929 02:53:02.825176 16523 tablet_bootstrap.cc:492] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: No bootstrap required, opened a new log
I20260929 02:53:02.825342 16523 ts_tablet_manager.cc:1397] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260929 02:53:02.825356 16522 consensus_queue.cc:260] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.826228 16523 raft_consensus.cc:348] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.826463 16523 raft_consensus.cc:374] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:02.826567 16523 raft_consensus.cc:729] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Initialized, Role: FOLLOWER
I20260929 02:53:02.826880 16523 consensus_queue.cc:260] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.828451 16523 ts_tablet_manager.cc:1428] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260929 02:53:02.830574 16522 ts_tablet_manager.cc:1428] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260929 02:53:02.831333 16546 ts_tablet_manager.cc:1918] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:02.831521 16546 log.cc:1196] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/31c7d4e27a0047e8adeab5d141a55cfa
I20260929 02:53:02.832130 16546 ts_tablet_manager.cc:1939] T 31c7d4e27a0047e8adeab5d141a55cfa P fd980ab14f9b475a9345497ced4ed82e: Deleting consensus metadata
I20260929 02:53:02.833688 16000 catalog_manager.cc:4841] TS fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127): tablet 31c7d4e27a0047e8adeab5d141a55cfa (table test-table [id=7012a888e670431dadfa395752fdfbd3]) successfully deleted
I20260929 02:53:02.943348 16529 raft_consensus.cc:482] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:53:02.943636 16529 raft_consensus.cc:504] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.945451 16529 leader_election.cc:288] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555), fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541)
I20260929 02:53:02.953543 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.945744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:33555 (local address 127.8.57.3:46187)
0929 02:53:02.946145 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.946166 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.946289 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.947116 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.947122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.947149 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.947331 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.947342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.948865 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.948870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.950934 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.950947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.952745 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.952751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.952976 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.953011 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.953093 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.953231 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260929 02:53:02.954278 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.946533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:46187 (local address 127.8.57.1:33555)
0929 02:53:02.946705 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.946710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.946728 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:53:02.946795 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.946800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.946869 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.946958 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.947496 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.948705 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.951139 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.952408 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.952538 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.952653 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.953846 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.953990 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:53:02.954536 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.946246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:32833 (local address 127.8.57.2:38541)
0929 02:53:02.946400 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:02.946405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:02.946934 (+ 529us) server_negotiation.cc:407] Connection header received
0929 02:53:02.947176 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:02.947181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:02.947252 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:02.947307 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:02.948083 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.949164 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.951812 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.953052 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.953148 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.953244 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:02.954110 (+ 866us) server_negotiation.cc:299] Negotiation successful
0929 02:53:02.954247 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:53:02.955076 16175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a67a195074cf0bcda889f472a5e5f" candidate_uuid: "fd980ab14f9b475a9345497ced4ed82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec497747a1e41efaf027ab0424adf02" is_pre_election: true
I20260929 02:53:02.955572 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:02.945880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:38541 (local address 127.8.57.3:32833)
0929 02:53:02.946833 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:02.946850 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:02.947032 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:02.947517 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:02.947522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:02.947546 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:02.947864 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.947903 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.949339 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.949344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.951630 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:02.951644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:02.953472 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:02.953479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:02.953926 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:02.953962 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:53:02.955011 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0929 02:53:02.955157 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"threads_started":1}
I20260929 02:53:02.955564 16175 raft_consensus.cc:2454] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd980ab14f9b475a9345497ced4ed82e in term 0.
I20260929 02:53:02.956231 16381 leader_election.cc:302] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fd980ab14f9b475a9345497ced4ed82e; no voters:
I20260929 02:53:02.956728 16529 raft_consensus.cc:2781] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:53:02.956851 16529 raft_consensus.cc:482] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:53:02.956805 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a67a195074cf0bcda889f472a5e5f" candidate_uuid: "fd980ab14f9b475a9345497ced4ed82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf" is_pre_election: true
I20260929 02:53:02.956940 16529 raft_consensus.cc:3037] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:02.957047 16314 raft_consensus.cc:2454] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd980ab14f9b475a9345497ced4ed82e in term 0.
I20260929 02:53:02.960007 16529 raft_consensus.cc:504] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.961136 16175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a67a195074cf0bcda889f472a5e5f" candidate_uuid: "fd980ab14f9b475a9345497ced4ed82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec497747a1e41efaf027ab0424adf02"
I20260929 02:53:02.961369 16175 raft_consensus.cc:3037] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:02.961566 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a67a195074cf0bcda889f472a5e5f" candidate_uuid: "fd980ab14f9b475a9345497ced4ed82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf"
I20260929 02:53:02.961769 16314 raft_consensus.cc:3037] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:02.960909 16529 leader_election.cc:288] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [CANDIDATE]: Term 1 election: Requested vote from peers 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555), fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541)
I20260929 02:53:02.964190 16175 raft_consensus.cc:2454] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd980ab14f9b475a9345497ced4ed82e in term 1.
I20260929 02:53:02.964632 16381 leader_election.cc:302] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fd980ab14f9b475a9345497ced4ed82e; no voters:
I20260929 02:53:02.964684 16314 raft_consensus.cc:2454] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd980ab14f9b475a9345497ced4ed82e in term 1.
I20260929 02:53:02.965011 16529 raft_consensus.cc:2781] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:02.967507 16529 raft_consensus.cc:686] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 1 LEADER]: Becoming Leader. State: Replica: fd980ab14f9b475a9345497ced4ed82e, State: Running, Role: LEADER
I20260929 02:53:02.967900 16529 consensus_queue.cc:237] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:02.972780 16012 catalog_manager.cc:5495] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e reported cstate change: term changed from 0 to 1, leader changed from <none> to fd980ab14f9b475a9345497ced4ed82e (127.8.57.3). New cstate: current_term: 1 leader_uuid: "fd980ab14f9b475a9345497ced4ed82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } health_report { overall_health: HEALTHY } } }
W20260929 02:53:03.031862 16499 tablet.cc:2369] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:53:03.032020 16221 tablet.cc:2369] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:53:03.034472 16360 tablet.cc:2369] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:53:03 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650380, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.62@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:03.048779 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.030343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37087 (local address 127.0.0.1:56056)
0929 02:53:03.030975 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:03.030997 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:03.031168 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:03.031690 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:03.031694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:03.033260 (+ 1566us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:03.033659 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.033676 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.035132 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.035137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.036689 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.036703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.037695 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.037702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.037872 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.038668 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:03.038691 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:03.041724 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:03.045758 (+ 4034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:03.045773 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:03.045781 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:03.046388 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:03.046796 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:03.046801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:03.046804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:03.046989 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:03.047579 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:03.047587 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:03.047788 (+ 201us) client_negotiation.cc:769] Sending connection context
0929 02:53:03.048010 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 02:53:03.048333 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":775,"threads_started":1}
I20260929 02:53:03.049232 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.030343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.8.57.62:37087)
0929 02:53:03.030640 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.030646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.031099 (+ 453us) server_negotiation.cc:407] Connection header received
0929 02:53:03.031314 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.031320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.031390 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.031462 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:03.033855 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.034957 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.036870 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.037490 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.037635 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.042052 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:03.042059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:03.042064 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:03.042103 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:03.045583 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:03.046527 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:03.046533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:03.046536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:03.046680 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:03.047097 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:03.047102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:03.047105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:03.047431 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:03.047570 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.048140 (+ 570us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.048487 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":120}
I20260929 02:53:03.055022 16012 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:56056:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:53:03.056392 16012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:53:03.063607 16289 tablet_service.cc:1467] Processing CreateTablet for tablet 22574219e39e409a9ad6ed33ad6c4ddc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=97a7ba83366e48368e3f544fa6c3e6c7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:53:03.063545 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 22574219e39e409a9ad6ed33ad6c4ddc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=97a7ba83366e48368e3f544fa6c3e6c7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:53:03.064177 16289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22574219e39e409a9ad6ed33ad6c4ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.064181 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22574219e39e409a9ad6ed33ad6c4ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.068930 16428 tablet_service.cc:1467] Processing CreateTablet for tablet 22574219e39e409a9ad6ed33ad6c4ddc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=97a7ba83366e48368e3f544fa6c3e6c7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:53:03.069515 16428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22574219e39e409a9ad6ed33ad6c4ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.071346 16523 tablet_bootstrap.cc:492] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: Bootstrap starting.
I20260929 02:53:03.073212 16523 tablet_bootstrap.cc:654] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.074712 16522 tablet_bootstrap.cc:492] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: Bootstrap starting.
I20260929 02:53:03.076614 16522 tablet_bootstrap.cc:654] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.077000 16524 tablet_bootstrap.cc:492] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: Bootstrap starting.
I20260929 02:53:03.078693 16524 tablet_bootstrap.cc:654] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.079057 16522 tablet_bootstrap.cc:492] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: No bootstrap required, opened a new log
I20260929 02:53:03.079279 16522 ts_tablet_manager.cc:1397] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 02:53:03.080175 16522 raft_consensus.cc:348] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.080410 16522 raft_consensus.cc:374] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.080479 16522 raft_consensus.cc:729] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcce2a22b2c74854b42555a803cf08bf, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.080770 16522 consensus_queue.cc:260] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.080931 16524 tablet_bootstrap.cc:492] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: No bootstrap required, opened a new log
I20260929 02:53:03.081038 16523 tablet_bootstrap.cc:492] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: No bootstrap required, opened a new log
I20260929 02:53:03.081070 16524 ts_tablet_manager.cc:1397] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:53:03.081207 16523 ts_tablet_manager.cc:1397] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.003s
I20260929 02:53:03.081202 16528 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.081513 16528 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.081794 16524 raft_consensus.cc:348] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.082069 16524 raft_consensus.cc:374] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.082154 16524 raft_consensus.cc:729] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd980ab14f9b475a9345497ced4ed82e, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.082371 16524 consensus_queue.cc:260] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.082510 16523 raft_consensus.cc:348] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.082759 16529 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.083029 16529 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.083151 16523 raft_consensus.cc:374] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.083297 16523 raft_consensus.cc:729] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.083601 16523 consensus_queue.cc:260] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.083999 16524 ts_tablet_manager.cc:1428] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 02:53:03.084128 16530 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.084383 16530 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.084919 16522 ts_tablet_manager.cc:1428] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260929 02:53:03.085424 16523 ts_tablet_manager.cc:1428] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260929 02:53:03.207350 16530 raft_consensus.cc:482] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:53:03.207576 16530 raft_consensus.cc:504] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.208279 16530 leader_election.cc:288] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:03.208825 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22574219e39e409a9ad6ed33ad6c4ddc" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf" is_pre_election: true
I20260929 02:53:03.208935 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22574219e39e409a9ad6ed33ad6c4ddc" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e" is_pre_election: true
I20260929 02:53:03.209107 16314 raft_consensus.cc:2454] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:03.209182 16448 raft_consensus.cc:2454] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:03.209623 16103 leader_election.cc:302] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fd980ab14f9b475a9345497ced4ed82e; no voters:
I20260929 02:53:03.209903 16530 raft_consensus.cc:2781] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:53:03.210021 16530 raft_consensus.cc:482] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:53:03.210132 16530 raft_consensus.cc:3037] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.212221 16530 raft_consensus.cc:504] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.212710 16530 leader_election.cc:288] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Requested vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:03.213250 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22574219e39e409a9ad6ed33ad6c4ddc" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf"
I20260929 02:53:03.213318 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22574219e39e409a9ad6ed33ad6c4ddc" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e"
I20260929 02:53:03.213455 16314 raft_consensus.cc:3037] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.213518 16448 raft_consensus.cc:3037] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.215474 16448 raft_consensus.cc:2454] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:03.215464 16314 raft_consensus.cc:2454] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:03.215889 16103 leader_election.cc:302] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fcce2a22b2c74854b42555a803cf08bf; no voters:
I20260929 02:53:03.216189 16530 raft_consensus.cc:2781] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:03.216327 16530 raft_consensus.cc:686] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Running, Role: LEADER
I20260929 02:53:03.216588 16530 consensus_queue.cc:237] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.218175 16533 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: TxnStatusTablet state changed. Reason: New leader 7ec497747a1e41efaf027ab0424adf02. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.218441 16533 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:53:03.218999 16564 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:53:03.219210 16564 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:53:03.219671 16013 catalog_manager.cc:5495] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1). New cstate: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } health_report { overall_health: UNKNOWN } } }
I20260929 02:53:03.243466 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.238045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.8.57.1:33555)
0929 02:53:03.238213 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.238218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.238235 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:53:03.238490 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.238494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.238555 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.238642 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:03.239048 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.239898 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.241768 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.242208 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.242302 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.242822 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.242936 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.242998 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.243167 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.243234 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:53:03.243614 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.237908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:33555 (local address 127.0.0.1:59750)
0929 02:53:03.238143 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:03.238164 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:03.238305 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:03.238682 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:03.238687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:03.238701 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:53:03.238903 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.238914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.240040 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.240046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.241629 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.241646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.242370 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.242377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.242571 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.242656 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.243025 (+ 369us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.243053 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 02:53:03.243135 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:53:03.243300 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 02:53:03.247272 16448 raft_consensus.cc:1261] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.247388 16314 raft_consensus.cc:1261] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.247960 16533 consensus_queue.cc:1035] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.248216 16530 consensus_queue.cc:1035] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.251339 16529 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: TxnStatusTablet state changed. Reason: New leader 7ec497747a1e41efaf027ab0424adf02. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.251569 16529 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.252770 16528 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: TxnStatusTablet state changed. Reason: New leader 7ec497747a1e41efaf027ab0424adf02. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.253065 16528 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.255081 16533 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.255367 16533 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:53:03.256371 16536 mvcc.cc:205] Tried to move back new op lower bound from 7334503969775099904 to 7334503969657520128. Current Snapshot: MvccSnapshot[applied={T|T < 7334503969775099904 or (T in {7334503969775099904})}]
I20260929 02:53:03.258101 16538 mvcc.cc:205] Tried to move back new op lower bound from 7334503969775099904 to 7334503969657520128. Current Snapshot: MvccSnapshot[applied={T|T < 7334503969775099904 or (T in {7334503969775099904})}]
I20260929 02:53:03.258657 16529 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.258939 16529 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P fd980ab14f9b475a9345497ced4ed82e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.260723 16533 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.260962 16533 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P 7ec497747a1e41efaf027ab0424adf02: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:53:03.264644 16528 tablet_replica.cc:439] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } } }
I20260929 02:53:03.264901 16528 tablet_replica.cc:442] T 22574219e39e409a9ad6ed33ad6c4ddc P fcce2a22b2c74854b42555a803cf08bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:53:03.282510 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.276481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.8.57.3:33127)
0929 02:53:03.277128 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.277134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.277154 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:53:03.277258 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.277262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.277348 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.277499 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:03.277986 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.278743 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.280191 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.280607 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.280723 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.281181 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.281811 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.281958 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.282091 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.282191 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":349,"threads_started":1}
I20260929 02:53:03.299196 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.293404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:41473 (local address 127.8.57.1:33555)
0929 02:53:03.293578 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.293582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.294150 (+ 568us) server_negotiation.cc:407] Connection header received
0929 02:53:03.294342 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.294346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.294416 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.294497 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:03.294958 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.295673 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.297189 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.297797 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.297916 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.298372 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.298508 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.298587 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.298735 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.298839 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:53:03.299216 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.293399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:33555 (local address 127.8.57.3:41473)
0929 02:53:03.294039 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:03.294055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:03.294206 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:03.294526 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:03.294531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:03.294542 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:53:03.294813 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.294822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.295795 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.295801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.297044 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.297058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.298001 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.298009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.298142 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.298202 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.298602 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.298629 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 02:53:03.298701 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:53:03.298845 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":340,"threads_started":1}
I20260929 02:53:03.309659 16314 raft_consensus.cc:1261] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Refusing update from remote peer fd980ab14f9b475a9345497ced4ed82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.310316 16175 raft_consensus.cc:1261] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Refusing update from remote peer fd980ab14f9b475a9345497ced4ed82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.310325 16552 consensus_queue.cc:1035] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.311005 16529 consensus_queue.cc:1035] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.346196 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.337758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:54307 (local address 127.8.57.3:33127)
0929 02:53:03.337899 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.337904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.340465 (+ 2561us) server_negotiation.cc:407] Connection header received
0929 02:53:03.340655 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.340661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.340731 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.340800 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:03.341309 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.342215 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.344129 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.344816 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.344939 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.345395 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.345531 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.345623 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.345810 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.345892 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 02:53:03.346375 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.337688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33127 (local address 127.8.57.1:54307)
0929 02:53:03.340365 (+ 2677us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:03.340381 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:03.340523 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:03.340831 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:03.340836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:03.340848 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:53:03.341155 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.341164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.342362 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.342365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.343989 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.344009 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.344985 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.344990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.345168 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.345232 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.345654 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.345667 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 02:53:03.345761 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 02:53:03.345897 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2504,"thread_start_us":2350,"threads_started":1}
I20260929 02:53:03.391717 16012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:03.392036 16012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:03.395610 16012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Sending DeleteTablet for 3 replicas of tablet 001a67a195074cf0bcda889f472a5e5f
I20260929 02:53:03.396545 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet 001a67a195074cf0bcda889f472a5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:59734
I20260929 02:53:03.396643 16289 tablet_service.cc:1514] Processing DeleteTablet for tablet 001a67a195074cf0bcda889f472a5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:51180
I20260929 02:53:03.404098 16428 tablet_service.cc:1514] Processing DeleteTablet for tablet 001a67a195074cf0bcda889f472a5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:46814
I20260929 02:53:03.408835 16583 tablet_replica.cc:330] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: stopping tablet replica
I20260929 02:53:03.409240 16583 raft_consensus.cc:2229] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:03.409639 16583 raft_consensus.cc:2258] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.410998 16583 ts_tablet_manager.cc:1905] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.419206 16583 ts_tablet_manager.cc:1918] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:53:03.419519 16583 log.cc:1196] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/001a67a195074cf0bcda889f472a5e5f
I20260929 02:53:03.420145 16583 ts_tablet_manager.cc:1939] T 001a67a195074cf0bcda889f472a5e5f P 7ec497747a1e41efaf027ab0424adf02: Deleting consensus metadata
I20260929 02:53:03.421432 15999 catalog_manager.cc:4841] TS 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555): tablet 001a67a195074cf0bcda889f472a5e5f (table test-table [id=0ef643a8e778491ab1a4fee54a4bcf3f]) successfully deleted
I20260929 02:53:03.428076 16585 tablet_replica.cc:330] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: stopping tablet replica
I20260929 02:53:03.428650 16584 tablet_replica.cc:330] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: stopping tablet replica
I20260929 02:53:03.428961 16585 raft_consensus.cc:2229] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:53:03.429255 16584 raft_consensus.cc:2229] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:03.429921 16585 raft_consensus.cc:2258] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.430253 16584 raft_consensus.cc:2258] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.431506 16585 ts_tablet_manager.cc:1905] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.440030 16585 ts_tablet_manager.cc:1918] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:53:03.440289 16585 log.cc:1196] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/001a67a195074cf0bcda889f472a5e5f
I20260929 02:53:03.440714 16584 ts_tablet_manager.cc:1905] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.440951 16585 ts_tablet_manager.cc:1939] T 001a67a195074cf0bcda889f472a5e5f P fd980ab14f9b475a9345497ced4ed82e: Deleting consensus metadata
I20260929 02:53:03.444329 16000 catalog_manager.cc:4841] TS fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127): tablet 001a67a195074cf0bcda889f472a5e5f (table test-table [id=0ef643a8e778491ab1a4fee54a4bcf3f]) successfully deleted
I20260929 02:53:03.447500 16584 ts_tablet_manager.cc:1918] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:53:03.447705 16584 log.cc:1196] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/001a67a195074cf0bcda889f472a5e5f
I20260929 02:53:03.448168 16584 ts_tablet_manager.cc:1939] T 001a67a195074cf0bcda889f472a5e5f P fcce2a22b2c74854b42555a803cf08bf: Deleting consensus metadata
I20260929 02:53:03.449347 16000 catalog_manager.cc:4841] TS fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541): tablet 001a67a195074cf0bcda889f472a5e5f (table test-table [id=0ef643a8e778491ab1a4fee54a4bcf3f]) successfully deleted
Sep 29 02:53:03 dist-test-slave-1dkt krb5kdc[15968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:53:03 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:03 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:53:03.477825 16108 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:03.467694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56070 (local address 127.8.57.62:37087)
0929 02:53:03.467921 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.467928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.468420 (+ 492us) server_negotiation.cc:407] Connection header received
0929 02:53:03.468653 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.468659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.468755 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.468827 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:03.470252 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.470943 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.472218 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.472628 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.472740 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.477552 (+ 4812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56070: BlockingRecv error: recv got EOF from 127.0.0.1:56070 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 29 02:53:03 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM
I20260929 02:53:03.494295 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.480715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.8.57.62:37087)
0929 02:53:03.480892 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.480898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.481546 (+ 648us) server_negotiation.cc:407] Connection header received
0929 02:53:03.481706 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.481711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.481775 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.481858 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:03.483056 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.483830 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.485212 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.485606 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.485722 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.489348 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:03.489353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:03.489356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:03.489391 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:03.491772 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:03.492417 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:03.492422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:03.492424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:03.492540 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:03.492904 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:03.492908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:03.492912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:03.493192 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:03.493421 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.493706 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.493919 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:53:03.497679 16012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:53:03.498281 16012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:53:03.505201 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 857f6ab83edd4da7a758c74971c36021 (DEFAULT_TABLE table=test-table [id=d725640b102c4407a6da389979182240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:03.505705 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 857f6ab83edd4da7a758c74971c36021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.505930 16289 tablet_service.cc:1467] Processing CreateTablet for tablet 857f6ab83edd4da7a758c74971c36021 (DEFAULT_TABLE table=test-table [id=d725640b102c4407a6da389979182240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:03.506342 16289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 857f6ab83edd4da7a758c74971c36021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.512331 16428 tablet_service.cc:1467] Processing CreateTablet for tablet 857f6ab83edd4da7a758c74971c36021 (DEFAULT_TABLE table=test-table [id=d725640b102c4407a6da389979182240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:03.512625 16523 tablet_bootstrap.cc:492] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Bootstrap starting.
I20260929 02:53:03.512777 16428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 857f6ab83edd4da7a758c74971c36021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.514256 16523 tablet_bootstrap.cc:654] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.521446 16522 tablet_bootstrap.cc:492] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Bootstrap starting.
I20260929 02:53:03.523255 16522 tablet_bootstrap.cc:654] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.527288 16524 tablet_bootstrap.cc:492] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Bootstrap starting.
I20260929 02:53:03.529007 16524 tablet_bootstrap.cc:654] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.550571 16524 tablet_bootstrap.cc:492] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: No bootstrap required, opened a new log
I20260929 02:53:03.550848 16524 ts_tablet_manager.cc:1397] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.010s
I20260929 02:53:03.551882 16524 raft_consensus.cc:348] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.552316 16524 raft_consensus.cc:374] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.553669 16524 raft_consensus.cc:729] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd980ab14f9b475a9345497ced4ed82e, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.554066 16524 consensus_queue.cc:260] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.559790 16522 tablet_bootstrap.cc:492] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: No bootstrap required, opened a new log
I20260929 02:53:03.560086 16522 ts_tablet_manager.cc:1397] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.005s
I20260929 02:53:03.561069 16522 raft_consensus.cc:348] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.561488 16522 raft_consensus.cc:374] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.561625 16522 raft_consensus.cc:729] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcce2a22b2c74854b42555a803cf08bf, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.561949 16522 consensus_queue.cc:260] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.566515 16522 ts_tablet_manager.cc:1428] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260929 02:53:03.569416 16524 ts_tablet_manager.cc:1428] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260929 02:53:03.571772 16523 tablet_bootstrap.cc:492] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: No bootstrap required, opened a new log
I20260929 02:53:03.571966 16523 ts_tablet_manager.cc:1397] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.024s
I20260929 02:53:03.572830 16523 raft_consensus.cc:348] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.573151 16523 raft_consensus.cc:374] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.573261 16523 raft_consensus.cc:729] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.573526 16523 consensus_queue.cc:260] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.574018 16523 ts_tablet_manager.cc:1428] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:53:03.705083 16568 raft_consensus.cc:482] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:53:03.705279 16568 raft_consensus.cc:504] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.705958 16568 leader_election.cc:288] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:03.706679 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "857f6ab83edd4da7a758c74971c36021" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf" is_pre_election: true
I20260929 02:53:03.706689 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "857f6ab83edd4da7a758c74971c36021" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e" is_pre_election: true
I20260929 02:53:03.706938 16314 raft_consensus.cc:2454] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:03.706944 16448 raft_consensus.cc:2454] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:03.707409 16103 leader_election.cc:302] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fd980ab14f9b475a9345497ced4ed82e; no voters:
I20260929 02:53:03.707672 16568 raft_consensus.cc:2781] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:53:03.707800 16568 raft_consensus.cc:482] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:53:03.707911 16568 raft_consensus.cc:3037] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.710839 16568 raft_consensus.cc:504] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.711524 16568 leader_election.cc:288] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Requested vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:03.712173 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "857f6ab83edd4da7a758c74971c36021" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf"
I20260929 02:53:03.712253 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "857f6ab83edd4da7a758c74971c36021" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e"
I20260929 02:53:03.712409 16314 raft_consensus.cc:3037] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.712455 16448 raft_consensus.cc:3037] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.715394 16314 raft_consensus.cc:2454] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:03.715411 16448 raft_consensus.cc:2454] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:03.715896 16103 leader_election.cc:302] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fcce2a22b2c74854b42555a803cf08bf; no voters:
I20260929 02:53:03.716221 16568 raft_consensus.cc:2781] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:03.716398 16568 raft_consensus.cc:686] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Running, Role: LEADER
I20260929 02:53:03.716684 16568 consensus_queue.cc:237] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.719727 16012 catalog_manager.cc:5495] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1). New cstate: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } health_report { overall_health: UNKNOWN } } }
I20260929 02:53:03.745811 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.740396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.8.57.1:33555)
0929 02:53:03.740730 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.740737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.740786 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:53:03.740861 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.740866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.740953 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.741047 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:03.741368 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.742461 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.743839 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.744486 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.744627 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.745041 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.745204 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.745298 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.745424 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.745512 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260929 02:53:03.748759 16314 raft_consensus.cc:1261] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.748821 16448 raft_consensus.cc:1261] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.749435 16568 consensus_queue.cc:1035] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.749722 16530 consensus_queue.cc:1035] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.755177 16537 mvcc.cc:205] Tried to move back new op lower bound from 7334503971829014528 to 7334503971706318848. Current Snapshot: MvccSnapshot[applied={T|T < 7334503971829014528}]
I20260929 02:53:03.763681 16012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:03.764055 16012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:03.767520 16012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Sending DeleteTablet for 3 replicas of tablet 857f6ab83edd4da7a758c74971c36021
I20260929 02:53:03.768420 16289 tablet_service.cc:1514] Processing DeleteTablet for tablet 857f6ab83edd4da7a758c74971c36021 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:51180
I20260929 02:53:03.768421 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet 857f6ab83edd4da7a758c74971c36021 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:59734
I20260929 02:53:03.768761 16583 tablet_replica.cc:330] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: stopping tablet replica
I20260929 02:53:03.768796 16428 tablet_service.cc:1514] Processing DeleteTablet for tablet 857f6ab83edd4da7a758c74971c36021 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:46814
I20260929 02:53:03.768975 16583 raft_consensus.cc:2229] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:53:03.769073 16585 tablet_replica.cc:330] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: stopping tablet replica
I20260929 02:53:03.769268 16585 raft_consensus.cc:2229] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:03.769323 16583 raft_consensus.cc:2258] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.769506 16585 raft_consensus.cc:2258] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.769618 16584 tablet_replica.cc:330] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: stopping tablet replica
I20260929 02:53:03.769814 16584 raft_consensus.cc:2229] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:03.770066 16584 raft_consensus.cc:2258] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.770622 16585 ts_tablet_manager.cc:1905] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.770601 16012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:53:03.771131 16012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:53:03.772570 16583 ts_tablet_manager.cc:1905] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.777257 16583 ts_tablet_manager.cc:1918] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:03.777405 16583 log.cc:1196] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/857f6ab83edd4da7a758c74971c36021
I20260929 02:53:03.777871 16583 ts_tablet_manager.cc:1939] T 857f6ab83edd4da7a758c74971c36021 P 7ec497747a1e41efaf027ab0424adf02: Deleting consensus metadata
I20260929 02:53:03.778743 16584 ts_tablet_manager.cc:1905] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.781291 16585 ts_tablet_manager.cc:1918] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:03.781585 16585 log.cc:1196] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/857f6ab83edd4da7a758c74971c36021
I20260929 02:53:03.782119 16585 ts_tablet_manager.cc:1939] T 857f6ab83edd4da7a758c74971c36021 P fd980ab14f9b475a9345497ced4ed82e: Deleting consensus metadata
I20260929 02:53:03.782866 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 993cda75117549fcb66fa5d077f8b26f (DEFAULT_TABLE table=test-table [id=eb74ab2f75c44ca6949d6aef168efba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:03.783345 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993cda75117549fcb66fa5d077f8b26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.785471 16289 tablet_service.cc:1467] Processing CreateTablet for tablet 993cda75117549fcb66fa5d077f8b26f (DEFAULT_TABLE table=test-table [id=eb74ab2f75c44ca6949d6aef168efba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:03.785851 16428 tablet_service.cc:1467] Processing CreateTablet for tablet 993cda75117549fcb66fa5d077f8b26f (DEFAULT_TABLE table=test-table [id=eb74ab2f75c44ca6949d6aef168efba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:03.785954 16289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993cda75117549fcb66fa5d077f8b26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.786244 16428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993cda75117549fcb66fa5d077f8b26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:03.787205 15999 catalog_manager.cc:4841] TS 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555): tablet 857f6ab83edd4da7a758c74971c36021 (table test-table [id=d725640b102c4407a6da389979182240]) successfully deleted
I20260929 02:53:03.790091 16523 tablet_bootstrap.cc:492] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Bootstrap starting.
I20260929 02:53:03.790318 16000 catalog_manager.cc:4841] TS fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127): tablet 857f6ab83edd4da7a758c74971c36021 (table test-table [id=d725640b102c4407a6da389979182240]) successfully deleted
I20260929 02:53:03.791648 16523 tablet_bootstrap.cc:654] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.791854 16522 tablet_bootstrap.cc:492] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Bootstrap starting.
I20260929 02:53:03.791812 16584 ts_tablet_manager.cc:1918] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:03.792092 16584 log.cc:1196] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/857f6ab83edd4da7a758c74971c36021
I20260929 02:53:03.792629 16584 ts_tablet_manager.cc:1939] T 857f6ab83edd4da7a758c74971c36021 P fcce2a22b2c74854b42555a803cf08bf: Deleting consensus metadata
I20260929 02:53:03.792668 16524 tablet_bootstrap.cc:492] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Bootstrap starting.
I20260929 02:53:03.793462 16522 tablet_bootstrap.cc:654] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.794297 16524 tablet_bootstrap.cc:654] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:03.796012 16522 tablet_bootstrap.cc:492] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: No bootstrap required, opened a new log
I20260929 02:53:03.796183 16522 ts_tablet_manager.cc:1397] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260929 02:53:03.796166 16000 catalog_manager.cc:4841] TS fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541): tablet 857f6ab83edd4da7a758c74971c36021 (table test-table [id=d725640b102c4407a6da389979182240]) successfully deleted
I20260929 02:53:03.796310 16523 tablet_bootstrap.cc:492] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: No bootstrap required, opened a new log
I20260929 02:53:03.796476 16523 ts_tablet_manager.cc:1397] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260929 02:53:03.796904 16522 raft_consensus.cc:348] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.797209 16522 raft_consensus.cc:374] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.797294 16522 raft_consensus.cc:729] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcce2a22b2c74854b42555a803cf08bf, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.797183 16523 raft_consensus.cc:348] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.797427 16523 raft_consensus.cc:374] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.797505 16523 raft_consensus.cc:729] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.797521 16522 consensus_queue.cc:260] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.797780 16523 consensus_queue.cc:260] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.798281 16523 ts_tablet_manager.cc:1428] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:53:03.798759 16522 ts_tablet_manager.cc:1428] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 02:53:03.802852 16524 tablet_bootstrap.cc:492] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: No bootstrap required, opened a new log
I20260929 02:53:03.803032 16524 ts_tablet_manager.cc:1397] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260929 02:53:03.803807 16524 raft_consensus.cc:348] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.804095 16524 raft_consensus.cc:374] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:03.804180 16524 raft_consensus.cc:729] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd980ab14f9b475a9345497ced4ed82e, State: Initialized, Role: FOLLOWER
I20260929 02:53:03.804432 16524 consensus_queue.cc:260] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.805027 16524 ts_tablet_manager.cc:1428] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 02:53:03.821106 16568 raft_consensus.cc:482] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:53:03.821336 16568 raft_consensus.cc:504] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.822155 16568 leader_election.cc:288] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:03.822643 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cda75117549fcb66fa5d077f8b26f" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf" is_pre_election: true
I20260929 02:53:03.822681 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cda75117549fcb66fa5d077f8b26f" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e" is_pre_election: true
I20260929 02:53:03.822902 16314 raft_consensus.cc:2454] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:03.822928 16448 raft_consensus.cc:2454] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 0.
I20260929 02:53:03.823329 16103 leader_election.cc:302] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fcce2a22b2c74854b42555a803cf08bf; no voters:
I20260929 02:53:03.823683 16568 raft_consensus.cc:2781] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:53:03.823796 16568 raft_consensus.cc:482] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:53:03.823902 16568 raft_consensus.cc:3037] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.825917 16568 raft_consensus.cc:504] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.826462 16568 leader_election.cc:288] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Requested vote from peers fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541), fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127)
I20260929 02:53:03.827060 16314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cda75117549fcb66fa5d077f8b26f" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce2a22b2c74854b42555a803cf08bf"
I20260929 02:53:03.827164 16448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cda75117549fcb66fa5d077f8b26f" candidate_uuid: "7ec497747a1e41efaf027ab0424adf02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd980ab14f9b475a9345497ced4ed82e"
I20260929 02:53:03.827275 16314 raft_consensus.cc:3037] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.827379 16448 raft_consensus.cc:3037] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:03.829864 16448 raft_consensus.cc:2454] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:03.830325 16103 leader_election.cc:302] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec497747a1e41efaf027ab0424adf02, fd980ab14f9b475a9345497ced4ed82e; no voters:
I20260929 02:53:03.830377 16314 raft_consensus.cc:2454] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec497747a1e41efaf027ab0424adf02 in term 1.
I20260929 02:53:03.831048 16568 raft_consensus.cc:2781] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:03.831182 16568 raft_consensus.cc:686] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec497747a1e41efaf027ab0424adf02, State: Running, Role: LEADER
I20260929 02:53:03.831611 16568 consensus_queue.cc:237] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } }
I20260929 02:53:03.834329 16012 catalog_manager.cc:5495] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1). New cstate: current_term: 1 leader_uuid: "7ec497747a1e41efaf027ab0424adf02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec497747a1e41efaf027ab0424adf02" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 33555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 } health_report { overall_health: UNKNOWN } } }
I20260929 02:53:03.867321 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.861510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:36421 (local address 127.8.57.1:33555)
0929 02:53:03.861690 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.861696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.862157 (+ 461us) server_negotiation.cc:407] Connection header received
0929 02:53:03.862290 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.862295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.862382 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.862482 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:03.862875 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.863935 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.865245 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.865923 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.866074 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.866473 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.866630 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.866721 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.866896 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.866986 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":37}
I20260929 02:53:03.867379 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.861500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:33555 (local address 127.8.57.1:36421)
0929 02:53:03.862049 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:03.862065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:03.862181 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:03.862484 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:03.862487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:03.862495 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:53:03.862731 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.862739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.864039 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.864043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.865103 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:03.865115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.866124 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.866129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:03.866274 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.866314 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:53:03.866748 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:53:03.866760 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 02:53:03.866848 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 02:53:03.866975 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":33,"thread_start_us":319,"threads_started":1}
I20260929 02:53:03.877781 16314 raft_consensus.cc:1261] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.877833 16448 raft_consensus.cc:1261] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Refusing update from remote peer 7ec497747a1e41efaf027ab0424adf02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:03.878381 16568 consensus_queue.cc:1035] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd980ab14f9b475a9345497ced4ed82e" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.878679 16530 consensus_queue.cc:1035] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcce2a22b2c74854b42555a803cf08bf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:03.937381 16012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:03.937633 16012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:03.940564 16012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94869980840c4e8e83d9f5c5b2af3c44: Sending DeleteTablet for 3 replicas of tablet 993cda75117549fcb66fa5d077f8b26f
I20260929 02:53:03.941325 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet 993cda75117549fcb66fa5d077f8b26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:59734
I20260929 02:53:03.941736 16583 tablet_replica.cc:330] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: stopping tablet replica
I20260929 02:53:03.941782 16289 tablet_service.cc:1514] Processing DeleteTablet for tablet 993cda75117549fcb66fa5d077f8b26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:51180
I20260929 02:53:03.941926 16583 raft_consensus.cc:2229] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:53:03.942071 16584 tablet_replica.cc:330] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: stopping tablet replica
I20260929 02:53:03.942239 16584 raft_consensus.cc:2229] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:03.942257 16583 raft_consensus.cc:2258] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.942446 16584 raft_consensus.cc:2258] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.943418 16584 ts_tablet_manager.cc:1905] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.943418 16583 ts_tablet_manager.cc:1905] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.948280 16428 tablet_service.cc:1514] Processing DeleteTablet for tablet 993cda75117549fcb66fa5d077f8b26f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:03 UTC) from {username='oryx', principal='oryx/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:46814
I20260929 02:53:03.948594 16585 tablet_replica.cc:330] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: stopping tablet replica
I20260929 02:53:03.948779 16585 raft_consensus.cc:2229] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:03.948987 16585 raft_consensus.cc:2258] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:03.950250 16584 ts_tablet_manager.cc:1918] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:53:03.950275 16585 ts_tablet_manager.cc:1905] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:03.950381 16584 log.cc:1196] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/993cda75117549fcb66fa5d077f8b26f
I20260929 02:53:03.950853 16584 ts_tablet_manager.cc:1939] T 993cda75117549fcb66fa5d077f8b26f P fcce2a22b2c74854b42555a803cf08bf: Deleting consensus metadata
I20260929 02:53:03.950907 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 16084
I20260929 02:53:03.952203 16000 catalog_manager.cc:4841] TS fcce2a22b2c74854b42555a803cf08bf (127.8.57.2:38541): tablet 993cda75117549fcb66fa5d077f8b26f (table test-table [id=eb74ab2f75c44ca6949d6aef168efba3]) successfully deleted
I20260929 02:53:03.955328 16583 ts_tablet_manager.cc:1918] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:53:03.955520 16583 log.cc:1196] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/993cda75117549fcb66fa5d077f8b26f
I20260929 02:53:03.956084 16583 ts_tablet_manager.cc:1939] T 993cda75117549fcb66fa5d077f8b26f P 7ec497747a1e41efaf027ab0424adf02: Deleting consensus metadata
I20260929 02:53:03.957471 16585 ts_tablet_manager.cc:1918] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:53:03.957484 15999 catalog_manager.cc:4841] TS 7ec497747a1e41efaf027ab0424adf02 (127.8.57.1:33555): tablet 993cda75117549fcb66fa5d077f8b26f (table test-table [id=eb74ab2f75c44ca6949d6aef168efba3]) successfully deleted
I20260929 02:53:03.957628 16585 log.cc:1196] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/993cda75117549fcb66fa5d077f8b26f
I20260929 02:53:03.958027 16585 ts_tablet_manager.cc:1939] T 993cda75117549fcb66fa5d077f8b26f P fd980ab14f9b475a9345497ced4ed82e: Deleting consensus metadata
I20260929 02:53:03.959019 16000 catalog_manager.cc:4841] TS fd980ab14f9b475a9345497ced4ed82e (127.8.57.3:33127): tablet 993cda75117549fcb66fa5d077f8b26f (table test-table [id=eb74ab2f75c44ca6949d6aef168efba3]) successfully deleted
Sep 29 02:53:03 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.1@KRBTEST.COM
I20260929 02:53:03.973029 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:03.953718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.8.57.1:33555)
0929 02:53:03.953907 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:03.953913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:03.957851 (+ 3938us) server_negotiation.cc:407] Connection header received
0929 02:53:03.958066 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:03.958071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:03.958204 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:03.958304 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:03.959897 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.960658 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.962360 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:03.962778 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:03.962876 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:03.967585 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:03.967601 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:03.967624 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:03.967663 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:03.970403 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:03.971199 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:03.971206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:03.971208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:03.971339 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:03.971690 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:03.971693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:03.971694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:03.972079 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:03.972309 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:03.972598 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 02:53:03.972776 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:53:03.973785 16215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:04.142261 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 16084
I20260929 02:53:04.164059 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 16224
Sep 29 02:53:04 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.2@KRBTEST.COM
I20260929 02:53:04.178449 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:04.164829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.8.57.2:38541)
0929 02:53:04.165311 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:04.165315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:04.165329 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:53:04.165380 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:04.165384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:04.165437 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:04.165529 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:04.166697 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:04.167438 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:04.168799 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:04.169237 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:04.169346 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:04.173156 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:04.173170 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:04.173185 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:04.173225 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:04.175845 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:04.176512 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:04.176520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:04.176522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:04.176659 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:04.177002 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:04.177006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:04.177008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:04.177389 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:04.177690 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:04.177937 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:53:04.178116 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":247,"threads_started":1}
I20260929 02:53:04.179188 16354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:04.276767 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 16224
I20260929 02:53:04.290571 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 16363
Sep 29 02:53:04 dist-test-slave-1dkt krb5kdc[15968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.57.3@KRBTEST.COM
I20260929 02:53:04.306006 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:04.291283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46838 (local address 127.8.57.3:33127)
0929 02:53:04.291766 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:04.291771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:04.291787 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:53:04.291842 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:04.291845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:04.291942 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:04.292035 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:04.293301 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:04.294011 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:04.295335 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:04.295788 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:04.296062 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:04.300318 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:04.300335 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:04.300351 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:04.300392 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:04.303222 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:04.303999 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:04.304009 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:04.304012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:04.304157 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:04.304572 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:04.304576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:04.304578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:04.304928 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:04.305185 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:04.305407 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 02:53:04.305640 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":258,"threads_started":1}
I20260929 02:53:04.306599 16492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:04.427980 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 16363
I20260929 02:53:04.444890 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 15981
I20260929 02:53:04.446079 16053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:04.562999 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 15981
2026-09-29T02:53:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5236 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:53:04 dist-test-slave-1dkt krb5kdc[16625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:53:04 dist-test-slave-1dkt krb5kdc[16625](info): set up 2 sockets
Sep 29 02:53:04 dist-test-slave-1dkt krb5kdc[16625](info): commencing operation
krb5kdc: starting...
W20260929 02:53:05.584730 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:53:05 dist-test-slave-1dkt krb5kdc[16625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:53:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:53:05.783412 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41523
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:06.096326 16638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:06.096650 16638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:06.096784 16638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:06.104084 16638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:06.104218 16638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:06.104300 16638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:06.104373 16638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:06.104442 16638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:06.114405 16638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:06.115705 16638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:06.117048 16638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:06.129091 16644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:06.129333 16646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:06.129597 16643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:06.129987 16638 server_base.cc:1034] running on GCE node
I20260929 02:53:06.130838 16638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:06.132481 16638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:06.133709 16638 hybrid_clock.cc:648] HybridClock initialized: now 1790650386133651 us; error 47 us; skew 500 ppm
I20260929 02:53:06.134105 16638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T02:53:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1640 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:53:06 dist-test-slave-1dkt krb5kdc[16652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:53:06 dist-test-slave-1dkt krb5kdc[16652](info): set up 2 sockets
Sep 29 02:53:06 dist-test-slave-1dkt krb5kdc[16652](info): commencing operation
krb5kdc: starting...
W20260929 02:53:07.204916 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:53:07 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:53:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:53:07.402951 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41517
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:07.707672 16665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:07.708009 16665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:07.708149 16665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:07.715341 16665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:07.715467 16665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:07.715547 16665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:07.715622 16665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:07.715683 16665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:07.725481 16665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:07.726994 16665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:07.728560 16665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:07.739962 16673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:07.740005 16671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:07.740442 16670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:07.741189 16665 server_base.cc:1034] running on GCE node
I20260929 02:53:07.742027 16665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:07.743549 16665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:07.744776 16665 hybrid_clock.cc:648] HybridClock initialized: now 1790650387744729 us; error 37 us; skew 500 ppm
I20260929 02:53:07.745144 16665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:07 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650387, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:07.748257 16665 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:53:07.749723 16665 webserver.cc:460] Webserver started at http://127.8.57.62:37761/ using document root <none> and password file <none>
I20260929 02:53:07.750437 16665 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:07.750537 16665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:07.750833 16665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:07.753787 16665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "de94ececaa8d45a7be10ee614e34b196"
format_stamp: "Formatted at 2026-09-29 02:53:07 on dist-test-slave-1dkt"
server_key: "23c0c8105c699df8f010400792ce1990"
server_key_iv: "1991e87a5802e351e53e749710c64a0d"
server_key_version: "encryptionkey@0"
I20260929 02:53:07.754534 16665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "de94ececaa8d45a7be10ee614e34b196"
format_stamp: "Formatted at 2026-09-29 02:53:07 on dist-test-slave-1dkt"
server_key: "23c0c8105c699df8f010400792ce1990"
server_key_iv: "1991e87a5802e351e53e749710c64a0d"
server_key_version: "encryptionkey@0"
I20260929 02:53:07.759415 16665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:53:07.762662 16680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:07.764091 16665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:07.764252 16665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "de94ececaa8d45a7be10ee614e34b196"
format_stamp: "Formatted at 2026-09-29 02:53:07 on dist-test-slave-1dkt"
server_key: "23c0c8105c699df8f010400792ce1990"
server_key_iv: "1991e87a5802e351e53e749710c64a0d"
server_key_version: "encryptionkey@0"
I20260929 02:53:07.764425 16665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:07.781524 16665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:07.785302 16665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:07.785552 16665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:07.801664 16665 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41517
I20260929 02:53:07.801694 16731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41517 every 8 connection(s)
I20260929 02:53:07.802901 16665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:53:07.806381 16732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:07.811195 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 16665
I20260929 02:53:07.811367 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:53:07.811961 8420 external_mini_cluster.cc:1428] Setting key 09eae23a7643b7d2da3a6a2db8e433ba
I20260929 02:53:07.816442 16732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Bootstrap starting.
I20260929 02:53:07.819751 16732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:07.820917 16732 log.cc:824] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:07.823762 16732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: No bootstrap required, opened a new log
Sep 29 02:53:07 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:07.832490 16732 raft_consensus.cc:348] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:07.832983 16732 raft_consensus.cc:374] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:07.833115 16732 raft_consensus.cc:729] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de94ececaa8d45a7be10ee614e34b196, State: Initialized, Role: FOLLOWER
I20260929 02:53:07.833926 16732 consensus_queue.cc:260] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:07.834156 16732 raft_consensus.cc:388] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:53:07.834323 16732 raft_consensus.cc:482] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:53:07.834507 16732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:07.835419 16735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:07.813762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.8.57.62:41517)
0929 02:53:07.814559 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:07.814569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:07.814609 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:53:07.816292 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:07.816312 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:07.816709 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:07.816999 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:07.818668 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:07.819964 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:07.821644 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:07.822450 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:07.822570 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:07.827010 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:07.827017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:07.827034 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:07.827066 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:07.830119 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:07.830979 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:07.830985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:07.830991 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:07.831189 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:07.831747 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:07.831752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:07.831756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:07.832429 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:07.832701 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:07.833305 (+ 604us) server_negotiation.cc:299] Negotiation successful
0929 02:53:07.833746 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":351,"threads_started":1}
I20260929 02:53:07.837735 16732 raft_consensus.cc:504] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:07.838272 16732 leader_election.cc:302] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de94ececaa8d45a7be10ee614e34b196; no voters:
I20260929 02:53:07.838924 16732 leader_election.cc:288] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:53:07.839037 16737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:07.839730 16737 raft_consensus.cc:686] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 LEADER]: Becoming Leader. State: Replica: de94ececaa8d45a7be10ee614e34b196, State: Running, Role: LEADER
I20260929 02:53:07.840325 16737 consensus_queue.cc:237] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:07.840997 16732 sys_catalog.cc:564] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:53:07.843056 16738 sys_catalog.cc:455] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de94ececaa8d45a7be10ee614e34b196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } } }
I20260929 02:53:07.843425 16738 sys_catalog.cc:458] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:07.844069 16739 sys_catalog.cc:455] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de94ececaa8d45a7be10ee614e34b196. Latest consensus state: current_term: 1 leader_uuid: "de94ececaa8d45a7be10ee614e34b196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } } }
I20260929 02:53:07.844376 16739 sys_catalog.cc:458] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:07.849061 16744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:53:07.853262 16744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:53:07.861415 16744 catalog_manager.cc:1347] Generated new cluster ID: 625e7a3552894692beca2516d070187a
I20260929 02:53:07.861526 16744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:53:07.872936 16744 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:53:07.874256 16744 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:53:07.883100 16744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Generated new TSK 0
I20260929 02:53:07.883992 16744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:53:07.933370 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41517
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:08.244567 16758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:08.244830 16758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:08.244925 16758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:08.252134 16758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:08.252233 16758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:08.252347 16758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:53:08.261637 16758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:08.262904 16758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:08.264155 16758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:08.280321 16763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:08.280254 16764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:08.280273 16766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:08.281160 16758 server_base.cc:1034] running on GCE node
I20260929 02:53:08.281731 16758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:08.282637 16758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:08.283833 16758 hybrid_clock.cc:648] HybridClock initialized: now 1790650388283787 us; error 43 us; skew 500 ppm
I20260929 02:53:08.284132 16758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:08 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650388, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:08.287004 16758 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:53:08.288359 16758 webserver.cc:460] Webserver started at http://127.8.57.1:42047/ using document root <none> and password file <none>
I20260929 02:53:08.288990 16758 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:08.289100 16758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:08.289391 16758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:08.292191 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "ec5a800cc9154e74a96cbfa2a9343c43"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "868bba3708209929127613d5fdc8db20"
server_key_iv: "aa23708bc0edc3a02194cd7cc700ddc3"
server_key_version: "encryptionkey@0"
I20260929 02:53:08.292907 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "ec5a800cc9154e74a96cbfa2a9343c43"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "868bba3708209929127613d5fdc8db20"
server_key_iv: "aa23708bc0edc3a02194cd7cc700ddc3"
server_key_version: "encryptionkey@0"
I20260929 02:53:08.297775 16758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:53:08.301015 16773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:08.302266 16758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:53:08.302407 16758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "ec5a800cc9154e74a96cbfa2a9343c43"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "868bba3708209929127613d5fdc8db20"
server_key_iv: "aa23708bc0edc3a02194cd7cc700ddc3"
server_key_version: "encryptionkey@0"
I20260929 02:53:08.302578 16758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:08.316454 16758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:08.319984 16758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:08.320245 16758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:08.321132 16758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:08.322343 16758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:08.322414 16758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:08.322518 16758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:08.322569 16758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:08.350570 16758 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:38573
I20260929 02:53:08.350611 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:38573 every 8 connection(s)
I20260929 02:53:08.351740 16758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:53:08.361944 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 16758
I20260929 02:53:08.362130 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:53:08.362730 8420 external_mini_cluster.cc:1428] Setting key aca1901d220ab303385c39ffd7e2f10a
Sep 29 02:53:08 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650388, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:08.370383 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:08.354016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:56931 (local address 127.8.57.62:41517)
0929 02:53:08.354421 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:08.354425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:08.355213 (+ 788us) server_negotiation.cc:407] Connection header received
0929 02:53:08.356964 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:08.356969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:08.357043 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:08.357134 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:08.358516 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.359200 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.360696 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.361093 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.361204 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:08.364564 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:08.364569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:08.364573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:08.364609 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:08.367554 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:08.368241 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:08.368246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:08.368250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:08.368374 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:08.368776 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:08.368780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:08.368782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:08.369092 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:08.369224 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:08.369829 (+ 605us) server_negotiation.cc:299] Negotiation successful
0929 02:53:08.370029 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":214,"threads_started":1}
I20260929 02:53:08.371157 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:08.354410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:56931)
0929 02:53:08.355063 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:08.355098 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:08.356713 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:08.357358 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:08.357365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:08.357839 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:08.358357 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:08.358373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.359339 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.359342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:08.360559 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:08.360571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.361294 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.361301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:08.361538 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:08.362175 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:08.362195 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:08.364321 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:08.367679 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:08.367685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:08.367697 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:08.368135 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:08.368480 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:08.368485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:08.368489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:08.368685 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:08.369199 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:08.369204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:08.369501 (+ 297us) client_negotiation.cc:769] Sending connection context
0929 02:53:08.369723 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 02:53:08.370083 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":247,"threads_started":1}
I20260929 02:53:08.372612 16886 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41517
I20260929 02:53:08.372897 16886 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:08.373548 16886 heartbeater.cc:511] Master 127.8.57.62:41517 requested a full tablet report, sending...
I20260929 02:53:08.375418 16697 ts_manager.cc:194] Registered new tserver with Master: ec5a800cc9154e74a96cbfa2a9343c43 (127.8.57.1:38573)
I20260929 02:53:08.376859 16697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:56931
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:53:08.407181 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41517
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:08.710165 16892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:08.710443 16892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:08.710574 16892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:08.717693 16892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:08.717813 16892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:08.717963 16892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:53:08.727348 16892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:08.728598 16892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:08.729853 16892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:08.742388 16897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:08.742404 16898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:08.742383 16900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:08.743499 16892 server_base.cc:1034] running on GCE node
I20260929 02:53:08.744164 16892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:08.745070 16892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:08.746263 16892 hybrid_clock.cc:648] HybridClock initialized: now 1790650388746218 us; error 33 us; skew 500 ppm
I20260929 02:53:08.746560 16892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:08 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650388, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:08.749514 16892 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:53:08.750952 16892 webserver.cc:460] Webserver started at http://127.8.57.2:40489/ using document root <none> and password file <none>
I20260929 02:53:08.751603 16892 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:08.751715 16892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:08.752038 16892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:08.754808 16892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "f0b4c2c8191b417e9f898fcd97163f31"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "2904a8dbf625b728d0190109bee32fee"
server_key_iv: "f7aa9012f42b7ccbe525bf0cea63e86d"
server_key_version: "encryptionkey@0"
I20260929 02:53:08.755587 16892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "f0b4c2c8191b417e9f898fcd97163f31"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "2904a8dbf625b728d0190109bee32fee"
server_key_iv: "f7aa9012f42b7ccbe525bf0cea63e86d"
server_key_version: "encryptionkey@0"
I20260929 02:53:08.760444 16892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:53:08.763691 16907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:08.765000 16892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:08.765146 16892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "f0b4c2c8191b417e9f898fcd97163f31"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "2904a8dbf625b728d0190109bee32fee"
server_key_iv: "f7aa9012f42b7ccbe525bf0cea63e86d"
server_key_version: "encryptionkey@0"
I20260929 02:53:08.765329 16892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:08.787914 16892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:08.791702 16892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:08.791970 16892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:08.792863 16892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:08.794102 16892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:08.794178 16892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:08.794284 16892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:08.794333 16892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:08.823977 16892 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:40571
I20260929 02:53:08.824033 17019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:40571 every 8 connection(s)
I20260929 02:53:08.825137 16892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:53:08.834348 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 16892
I20260929 02:53:08.834481 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:53:08.834898 8420 external_mini_cluster.cc:1428] Setting key 032e82f1dc0f9d02fa332b2394c905c4
Sep 29 02:53:08 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650388, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:08.843263 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:08.827364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54913 (local address 127.8.57.62:41517)
0929 02:53:08.827513 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:08.827517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:08.828616 (+ 1099us) server_negotiation.cc:407] Connection header received
0929 02:53:08.830334 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:08.830340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:08.830419 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:08.830501 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:08.831913 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.832593 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.834063 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.834455 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.834564 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:08.837895 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:08.837900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:08.837903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:08.837931 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:08.840241 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:08.840948 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:08.840954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:08.840957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:08.841108 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:08.841500 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:08.841504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:08.841507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:08.841835 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:08.841978 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:08.842711 (+ 733us) server_negotiation.cc:299] Negotiation successful
0929 02:53:08.842929 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:53:08.844447 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:08.827780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:54913)
0929 02:53:08.828467 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:08.828508 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:08.830086 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:08.830730 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:08.830738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:08.831208 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:08.831742 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:08.831764 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.832731 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.832734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:08.833918 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:08.833929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:08.834686 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:08.834694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:08.834892 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:08.835485 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:08.835506 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:08.837703 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:08.840356 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:08.840364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:08.840381 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:08.840845 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:08.841216 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:08.841220 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:08.841223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:08.841417 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:08.841957 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:08.841964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:08.842333 (+ 369us) client_negotiation.cc:769] Sending connection context
0929 02:53:08.842622 (+ 289us) client_negotiation.cc:240] Negotiation successful
0929 02:53:08.843139 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":250,"threads_started":1}
I20260929 02:53:08.845767 17020 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41517
I20260929 02:53:08.846073 17020 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:08.846864 17020 heartbeater.cc:511] Master 127.8.57.62:41517 requested a full tablet report, sending...
I20260929 02:53:08.848255 16697 ts_manager.cc:194] Registered new tserver with Master: f0b4c2c8191b417e9f898fcd97163f31 (127.8.57.2:40571)
I20260929 02:53:08.849103 16697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:54913
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:53:08.880385 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41517
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:09.188966 17025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:09.189262 17025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:09.189390 17025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:09.196475 17025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:09.196589 17025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:09.196746 17025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:53:09.206185 17025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:09.207414 17025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:09.208663 17025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:09.220652 17030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:09.220604 17031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:09.220804 17033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:09.221613 17025 server_base.cc:1034] running on GCE node
I20260929 02:53:09.222190 17025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:09.223043 17025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:09.224228 17025 hybrid_clock.cc:648] HybridClock initialized: now 1790650389224185 us; error 37 us; skew 500 ppm
I20260929 02:53:09.224514 17025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:09 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650389, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:09.227332 17025 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:53:09.228616 17025 webserver.cc:460] Webserver started at http://127.8.57.3:36671/ using document root <none> and password file <none>
I20260929 02:53:09.229238 17025 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:09.229341 17025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:09.229609 17025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:09.232345 17025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "cc7c750826be41d68a84a8f4b5acfca0"
format_stamp: "Formatted at 2026-09-29 02:53:09 on dist-test-slave-1dkt"
server_key: "7baa20dd71d1274143514873ffae4dc7"
server_key_iv: "bd7061c70d29e67db32d3ceb1ee25fb0"
server_key_version: "encryptionkey@0"
I20260929 02:53:09.233067 17025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "cc7c750826be41d68a84a8f4b5acfca0"
format_stamp: "Formatted at 2026-09-29 02:53:09 on dist-test-slave-1dkt"
server_key: "7baa20dd71d1274143514873ffae4dc7"
server_key_iv: "bd7061c70d29e67db32d3ceb1ee25fb0"
server_key_version: "encryptionkey@0"
I20260929 02:53:09.237963 17025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:53:09.241307 17040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:09.242730 17025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:53:09.242879 17025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "cc7c750826be41d68a84a8f4b5acfca0"
format_stamp: "Formatted at 2026-09-29 02:53:09 on dist-test-slave-1dkt"
server_key: "7baa20dd71d1274143514873ffae4dc7"
server_key_iv: "bd7061c70d29e67db32d3ceb1ee25fb0"
server_key_version: "encryptionkey@0"
I20260929 02:53:09.243086 17025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:09.258221 17025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:09.262063 17025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:09.262322 17025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:09.263268 17025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:09.264552 17025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:09.264626 17025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:09.264750 17025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:09.264812 17025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:09.293844 17025 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:41035
I20260929 02:53:09.293887 17152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:41035 every 8 connection(s)
I20260929 02:53:09.295056 17025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:53:09.298815 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17025
I20260929 02:53:09.298974 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:53:09.299388 8420 external_mini_cluster.cc:1428] Setting key 51800af75bfb0d6b697b6259d58467ed
Sep 29 02:53:09 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650389, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:09.313688 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:09.297425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:40625 (local address 127.8.57.62:41517)
0929 02:53:09.297596 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:09.297601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:09.298755 (+ 1154us) server_negotiation.cc:407] Connection header received
0929 02:53:09.300850 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:09.300855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:09.300938 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:09.301021 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:09.302627 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.303336 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.304866 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.305261 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.305382 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:09.308590 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:09.308594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:09.308598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:09.308630 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:09.310876 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.311523 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.311527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.311530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.311637 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.312068 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.312072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.312076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.312329 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:09.312453 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:09.313128 (+ 675us) server_negotiation.cc:299] Negotiation successful
0929 02:53:09.313341 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:53:09.314765 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:09.297879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:40625)
0929 02:53:09.298596 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:09.298634 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:09.300573 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:09.301293 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:09.301305 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:09.301842 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:09.302477 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:09.302494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.303481 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.303484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:09.304739 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:09.304750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.305470 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.305476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:09.305710 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:09.306307 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:09.306327 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:09.308395 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:09.310997 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:09.311005 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:09.311022 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:09.311411 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:09.311738 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:09.311743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:09.311747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:09.311967 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:09.312443 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:09.312450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:09.312779 (+ 329us) client_negotiation.cc:769] Sending connection context
0929 02:53:09.313005 (+ 226us) client_negotiation.cc:240] Negotiation successful
0929 02:53:09.313485 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":270,"threads_started":1}
I20260929 02:53:09.316071 17153 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41517
I20260929 02:53:09.316329 17153 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:09.316973 17153 heartbeater.cc:511] Master 127.8.57.62:41517 requested a full tablet report, sending...
I20260929 02:53:09.318239 16697 ts_manager.cc:194] Registered new tserver with Master: cc7c750826be41d68a84a8f4b5acfca0 (127.8.57.3:41035)
I20260929 02:53:09.318966 16697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:40625
I20260929 02:53:09.326543 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab.
Entry for principal oryx/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab.
I20260929 02:53:09.379669 16886 heartbeater.cc:503] Master 127.8.57.62:41517 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab.
Entry for principal oryx/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab.
WARNING: no policy specified for oryx/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab.
Entry for principal oryx/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab.
I20260929 02:53:09.451000 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 16758
Sep 29 02:53:09 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:53:09.467021 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:09.452062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.8.57.1:38573)
0929 02:53:09.453336 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:09.453344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:09.453368 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:53:09.453430 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:09.453437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:09.453638 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:09.453767 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:09.455074 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.456114 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.457361 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.457890 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.457977 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:09.461861 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:09.461874 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:09.461889 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:09.461925 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:09.464578 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.465237 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.465243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.465245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.465363 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.465706 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.465709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.465711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.466047 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:09.466264 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:09.466552 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 02:53:09.466744 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1134,"thread_start_us":984,"threads_started":1}
I20260929 02:53:09.468385 16881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:09.549281 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 16758
I20260929 02:53:09.562219 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 16892
Sep 29 02:53:09 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:53:09.576828 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:09.563015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.8.57.2:40571)
0929 02:53:09.563549 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:09.563556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:09.563582 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:53:09.563641 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:09.563648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:09.563849 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:09.564013 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:09.565093 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.566121 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.567360 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.567930 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.568017 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:09.571861 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:09.571900 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:09.571916 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:09.571957 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:09.574547 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.575056 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.575062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.575064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.575191 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.575444 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.575447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.575449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.575766 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:09.576233 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:09.576355 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 02:53:09.576540 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":274,"threads_started":1}
I20260929 02:53:09.578023 17015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:09.658849 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 16892
I20260929 02:53:09.670718 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17025
Sep 29 02:53:09 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:53:09.685858 17167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:09.671580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.8.57.3:41035)
0929 02:53:09.672247 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:09.672253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:09.672281 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:53:09.672339 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:09.672345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:09.672542 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:09.672695 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:09.673906 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.674909 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.676183 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:09.676722 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:09.676831 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:09.680772 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:09.680783 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:09.680797 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:09.680829 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:09.683217 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.683808 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.683814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.683817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.684032 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:09.684377 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:09.684382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:09.684386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:09.684719 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:09.685009 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:09.685271 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 02:53:09.685515 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":364,"threads_started":1}
I20260929 02:53:09.687062 17148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:09.767946 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17025
I20260929 02:53:09.779561 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 16665
I20260929 02:53:09.780700 16727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:09.880775 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 16665
I20260929 02:53:09.896450 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41517
--webserver_interface=127.8.57.62
--webserver_port=37761
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:10.206869 17170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:10.207161 17170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:10.207293 17170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:10.214414 17170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:10.214527 17170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:10.214603 17170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:10.214674 17170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:10.214731 17170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:10.224551 17170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=37761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:10.225853 17170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:10.227180 17170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:10.238608 17176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:10.238586 17178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:10.238909 17175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:10.239424 17170 server_base.cc:1034] running on GCE node
I20260929 02:53:10.240206 17170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:10.241588 17170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:10.242815 17170 hybrid_clock.cc:648] HybridClock initialized: now 1790650390242751 us; error 51 us; skew 500 ppm
I20260929 02:53:10.243228 17170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650390, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:10.246512 17170 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:53:10.249029 17170 webserver.cc:460] Webserver started at http://127.8.57.62:37761/ using document root <none> and password file <none>
I20260929 02:53:10.249711 17170 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:10.249843 17170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:10.255344 17170 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:53:10.258260 17185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:10.259728 17170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 02:53:10.259938 17170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "de94ececaa8d45a7be10ee614e34b196"
format_stamp: "Formatted at 2026-09-29 02:53:07 on dist-test-slave-1dkt"
server_key: "23c0c8105c699df8f010400792ce1990"
server_key_iv: "1991e87a5802e351e53e749710c64a0d"
server_key_version: "encryptionkey@0"
I20260929 02:53:10.260911 17170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:10.288419 17170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:10.292691 17170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:10.292943 17170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:10.309777 17170 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41517
I20260929 02:53:10.309823 17236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41517 every 8 connection(s)
I20260929 02:53:10.311224 17170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:53:10.314813 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17170
I20260929 02:53:10.315573 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:38573
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:41517
--webserver_port=42047
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:53:10.317976 17237 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:53:10.321213 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Bootstrap starting.
I20260929 02:53:10.346966 17237 log.cc:824] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:10.357918 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:53:10.358530 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Bootstrap complete.
I20260929 02:53:10.371479 17237 raft_consensus.cc:348] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:10.372668 17237 raft_consensus.cc:729] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de94ececaa8d45a7be10ee614e34b196, State: Initialized, Role: FOLLOWER
I20260929 02:53:10.373392 17237 consensus_queue.cc:260] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:10.373668 17237 raft_consensus.cc:388] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:53:10.373782 17237 raft_consensus.cc:482] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:53:10.373955 17237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:53:10.379689 17237 raft_consensus.cc:504] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:10.380365 17237 leader_election.cc:302] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de94ececaa8d45a7be10ee614e34b196; no voters:
I20260929 02:53:10.380990 17237 leader_election.cc:288] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:53:10.381132 17241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:53:10.382158 17241 raft_consensus.cc:686] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [term 2 LEADER]: Becoming Leader. State: Replica: de94ececaa8d45a7be10ee614e34b196, State: Running, Role: LEADER
I20260929 02:53:10.382685 17241 consensus_queue.cc:237] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } }
I20260929 02:53:10.383064 17237 sys_catalog.cc:564] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:53:10.384981 17242 sys_catalog.cc:455] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "de94ececaa8d45a7be10ee614e34b196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } } }
I20260929 02:53:10.385056 17243 sys_catalog.cc:455] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de94ececaa8d45a7be10ee614e34b196. Latest consensus state: current_term: 2 leader_uuid: "de94ececaa8d45a7be10ee614e34b196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de94ececaa8d45a7be10ee614e34b196" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41517 } } }
I20260929 02:53:10.385353 17242 sys_catalog.cc:458] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:10.385351 17243 sys_catalog.cc:458] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:10.388324 17246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:53:10.394018 17246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:53:10.395397 17246 catalog_manager.cc:1259] Loaded cluster ID: 625e7a3552894692beca2516d070187a
I20260929 02:53:10.395524 17246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:53:10.399477 17246 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:53:10.401033 17246 catalog_manager.cc:5879] T 00000000000000000000000000000000 P de94ececaa8d45a7be10ee614e34b196: Loaded TSK: 0
I20260929 02:53:10.402691 17246 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:53:10.660567 17239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:10.660851 17239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:10.660969 17239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:10.668721 17239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:10.668833 17239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:10.668970 17239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:53:10.679059 17239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.1.keytab
--principal=oryx/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:38573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=42047
--tserver_master_addrs=127.8.57.62:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:10.680382 17239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:10.681625 17239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:10.694751 17263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:10.694720 17264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:10.696128 17266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:10.702088 17239 server_base.cc:1034] running on GCE node
I20260929 02:53:10.702741 17239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:10.703770 17239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:10.704982 17239 hybrid_clock.cc:648] HybridClock initialized: now 1790650390704924 us; error 56 us; skew 500 ppm
I20260929 02:53:10.705295 17239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650390, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:10.708487 17239 init.cc:377] Logged in from keytab as oryx/127.8.57.1@KRBTEST.COM (short username oryx)
I20260929 02:53:10.710033 17239 webserver.cc:460] Webserver started at http://127.8.57.1:42047/ using document root <none> and password file <none>
I20260929 02:53:10.710716 17239 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:10.710839 17239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:10.716187 17239 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:53:10.718814 17273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:10.720207 17239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:10.720353 17239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "ec5a800cc9154e74a96cbfa2a9343c43"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "868bba3708209929127613d5fdc8db20"
server_key_iv: "aa23708bc0edc3a02194cd7cc700ddc3"
server_key_version: "encryptionkey@0"
I20260929 02:53:10.721263 17239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:10.742187 17239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:10.746081 17239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:10.746328 17239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:10.747258 17239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:10.748592 17239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:10.748674 17239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:10.748780 17239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:10.748850 17239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:10.778172 17239 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:38573
I20260929 02:53:10.778224 17385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:38573 every 8 connection(s)
I20260929 02:53:10.779527 17239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:53:10.789052 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17239
I20260929 02:53:10.789798 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:40571
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:41517
--webserver_port=40489
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:10.798219 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:10.782286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:54427)
0929 02:53:10.783028 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:10.783078 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:10.784818 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:10.786114 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:10.786121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:10.786611 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:10.787127 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:10.787144 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.788878 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.788883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:10.790791 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:10.790808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.791928 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.791934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:10.792204 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:10.792869 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:10.792902 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:10.796904 (+ 4002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":303,"threads_started":1}
W20260929 02:53:10.798291 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:10.782237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:54427 (local address 127.8.57.62:41517)
0929 02:53:10.782906 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:10.782919 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:10.783226 (+ 307us) server_negotiation.cc:407] Connection header received
0929 02:53:10.785347 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:10.785360 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:10.785696 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:10.785980 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:10.787341 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.788720 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.790960 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.791671 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.791818 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:10.797312 (+ 5494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:54427: BlockingRecv error: recv got EOF from 127.8.57.1:54427 (error 108)
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":251,"threads_started":1}
W20260929 02:53:10.799494 17386 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:41517 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.57.62:41517: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:10.807551 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:10.800094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:60423)
0929 02:53:10.800233 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:10.800251 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:10.800338 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:10.800723 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:10.800728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:10.801078 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:10.801212 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:10.801220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.802208 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.802212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:10.803093 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:10.803111 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.803749 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.803755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:10.803948 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:10.804458 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:10.804475 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:10.807272 (+ 2797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 02:53:10.807682 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:10.800292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:60423 (local address 127.8.57.62:41517)
0929 02:53:10.800457 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:10.800462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:10.800477 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:53:10.800527 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:10.800531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:10.800585 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:10.800659 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:10.801342 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.802078 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.803225 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.803620 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.803721 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:10.807475 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:60423: BlockingRecv error: recv got EOF from 127.8.57.1:60423 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:10 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:10.815997 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:10.808317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:49579)
0929 02:53:10.808565 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:10.808581 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:10.808658 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:10.808957 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:10.808960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:10.809217 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:10.809349 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:10.809356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.810683 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.810687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:10.811568 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:10.811578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.812493 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.812497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:10.812652 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:10.813199 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:10.813216 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:10.815758 (+ 2542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260929 02:53:10.816285 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:10.808508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:49579 (local address 127.8.57.62:41517)
0929 02:53:10.808687 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:10.808693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:10.808710 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:53:10.808760 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:10.808765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:10.808847 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:10.808951 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:10.809471 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.810551 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.811687 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:10.812361 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:10.812473 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:10.816028 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:49579: BlockingRecv error: recv got EOF from 127.8.57.1:49579 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 02:53:10.816344 17386 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:53:11.100512 17390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:11.100787 17390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:11.100878 17390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:11.108029 17390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:11.108135 17390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:11.108259 17390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:53:11.117820 17390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.2.keytab
--principal=oryx/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:40571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=40489
--tserver_master_addrs=127.8.57.62:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:11.119061 17390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:11.120363 17390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:11.133178 17395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:11.133308 17396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:11.134004 17398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:11.134322 17390 server_base.cc:1034] running on GCE node
I20260929 02:53:11.135097 17390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:11.136322 17390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:11.137511 17390 hybrid_clock.cc:648] HybridClock initialized: now 1790650391137454 us; error 53 us; skew 500 ppm
I20260929 02:53:11.137867 17390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650391, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:11.141696 17390 init.cc:377] Logged in from keytab as oryx/127.8.57.2@KRBTEST.COM (short username oryx)
I20260929 02:53:11.143198 17390 webserver.cc:460] Webserver started at http://127.8.57.2:40489/ using document root <none> and password file <none>
I20260929 02:53:11.143852 17390 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:11.144023 17390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:11.149266 17390 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:53:11.151865 17405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:11.153138 17390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:11.153271 17390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "f0b4c2c8191b417e9f898fcd97163f31"
format_stamp: "Formatted at 2026-09-29 02:53:08 on dist-test-slave-1dkt"
server_key: "2904a8dbf625b728d0190109bee32fee"
server_key_iv: "f7aa9012f42b7ccbe525bf0cea63e86d"
server_key_version: "encryptionkey@0"
I20260929 02:53:11.154139 17390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:11.173080 17390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:11.176901 17390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:11.177119 17390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:11.178020 17390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:11.179261 17390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:11.179342 17390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:11.179415 17390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:11.179469 17390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:11.208887 17390 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:40571
I20260929 02:53:11.208932 17517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:40571 every 8 connection(s)
I20260929 02:53:11.210271 17390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:53:11.210726 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17390
I20260929 02:53:11.211402 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:41035
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:41517
--webserver_port=36671
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:53:11.225813 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.212851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41873 (local address 127.8.57.62:41517)
0929 02:53:11.213014 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.213019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.214304 (+ 1285us) server_negotiation.cc:407] Connection header received
0929 02:53:11.216111 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.216117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.216203 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.216288 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.217734 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.218592 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.220137 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.220542 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.220648 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.225531 (+ 4883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:41873: BlockingRecv error: recv got EOF from 127.8.57.2:41873 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:53:11.226481 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.213450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:41873)
0929 02:53:11.214148 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.214186 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.215807 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.216526 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.216534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.217026 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.217562 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.217579 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.218749 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.218754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.220010 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.220021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.220753 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.220759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.220974 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.221605 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.221625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.225302 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":273,"threads_started":1}
W20260929 02:53:11.227108 17518 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:41517 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.57.62:41517: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:11.235401 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.227668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:53949)
0929 02:53:11.227964 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.227980 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.228071 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.228368 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.228371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.228651 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.228789 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.228796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.230118 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.230121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.230975 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.230985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.231840 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.231843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.231996 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.232532 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.232548 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.235201 (+ 2653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 02:53:11.235805 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.227731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:53949 (local address 127.8.57.62:41517)
0929 02:53:11.227867 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.227890 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.228062 (+ 172us) server_negotiation.cc:407] Connection header received
0929 02:53:11.228165 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.228171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.228253 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.228362 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.228917 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.229995 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.231106 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.231717 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.231844 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.235432 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:53949: BlockingRecv error: recv got EOF from 127.8.57.2:53949 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:11.243566 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.236139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:52609)
0929 02:53:11.236313 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.236326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.236410 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.236682 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.236685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.236937 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.237067 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.237074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.238347 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.238350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.239237 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.239248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.240115 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.240118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.240242 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.240764 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.240779 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.243367 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260929 02:53:11.243806 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.236126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:52609 (local address 127.8.57.62:41517)
0929 02:53:11.236294 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.236299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.236368 (+ 69us) server_negotiation.cc:407] Connection header received
0929 02:53:11.236495 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.236500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.236571 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.236657 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.237167 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.238223 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.239349 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.239988 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.240114 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.243600 (+ 3486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:52609: BlockingRecv error: recv got EOF from 127.8.57.2:52609 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 02:53:11.243985 17518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:53:11.531261 17521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:11.531527 17521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:11.531657 17521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:11.538893 17521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:11.539007 17521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:11.539168 17521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:53:11.549083 17521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/krb5kdc/oryx_127.8.57.3.keytab
--principal=oryx/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:41035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=36671
--tserver_master_addrs=127.8.57.62:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:11.550365 17521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:11.551647 17521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:11.564307 17527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:11.564323 17526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:11.564795 17529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:11.565009 17521 server_base.cc:1034] running on GCE node
I20260929 02:53:11.565670 17521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:11.566610 17521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:11.567807 17521 hybrid_clock.cc:648] HybridClock initialized: now 1790650391567761 us; error 40 us; skew 500 ppm
I20260929 02:53:11.568126 17521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650391, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:11.571022 17521 init.cc:377] Logged in from keytab as oryx/127.8.57.3@KRBTEST.COM (short username oryx)
I20260929 02:53:11.572417 17521 webserver.cc:460] Webserver started at http://127.8.57.3:36671/ using document root <none> and password file <none>
I20260929 02:53:11.573071 17521 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:11.573199 17521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:11.578373 17521 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:53:11.580994 17536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:11.582238 17521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:11.582376 17521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "cc7c750826be41d68a84a8f4b5acfca0"
format_stamp: "Formatted at 2026-09-29 02:53:09 on dist-test-slave-1dkt"
server_key: "7baa20dd71d1274143514873ffae4dc7"
server_key_iv: "bd7061c70d29e67db32d3ceb1ee25fb0"
server_key_version: "encryptionkey@0"
I20260929 02:53:11.583302 17521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:11.606302 17521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:11.610193 17521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:11.610450 17521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:11.611435 17521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:11.612851 17521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:11.612959 17521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:11.613067 17521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:11.613135 17521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:11.641817 17521 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:41035
I20260929 02:53:11.641871 17648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:41035 every 8 connection(s)
I20260929 02:53:11.643290 17521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:53:11.651376 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17521
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:53:11.659168 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.645724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:34245 (local address 127.8.57.62:41517)
0929 02:53:11.645888 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.645892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.646997 (+ 1105us) server_negotiation.cc:407] Connection header received
0929 02:53:11.648846 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.648851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.648935 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.649019 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.650515 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.651279 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.653399 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.653789 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.653876 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.658894 (+ 5018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:34245: BlockingRecv error: recv got EOF from 127.8.57.3:34245 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:53:11.659677 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.646179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:34245)
0929 02:53:11.646843 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.646884 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.648581 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.649273 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.649281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.649792 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.650350 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.650368 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.651455 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.651461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.653264 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.653279 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.654013 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.654022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.654333 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.655011 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.655038 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.658672 (+ 3634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":255,"threads_started":1}
W20260929 02:53:11.660420 17649 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:41517 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.57.62:41517: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:11.668097 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.652431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36364 (local address 127.8.57.62:41517)
0929 02:53:11.653048 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.653052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.653066 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:53:11.653103 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.653107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.653164 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.653243 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.654898 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.655629 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.657002 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.657479 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.657578 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.660335 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:11.660345 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:11.660365 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:11.660406 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:11.663836 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:11.664496 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:11.664505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:11.664515 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:11.664779 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:11.666237 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:11.666242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:11.666245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:11.666822 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:11.667117 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:11.667409 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 02:53:11.667711 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":434,"threads_started":1}
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:11.669507 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.660959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:55871)
0929 02:53:11.661374 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.661391 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.661496 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.661777 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.661781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.662168 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.662334 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.662345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.663323 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.663328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.664617 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.664630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.665467 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.665473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.665651 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.666161 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.666182 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.669253 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305}
W20260929 02:53:11.669726 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.661024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:55871 (local address 127.8.57.62:41517)
0929 02:53:11.661178 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.661182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.661453 (+ 271us) server_negotiation.cc:407] Connection header received
0929 02:53:11.661594 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.661597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.661647 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.661735 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.662456 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.663195 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.664733 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.665319 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.665439 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.669480 (+ 4041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:55871: BlockingRecv error: recv got EOF from 127.8.57.3:55871 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:11.677820 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.670968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:38615)
0929 02:53:11.671116 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.671132 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.671206 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.671593 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.671597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.671907 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.672050 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.672058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.672978 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.672984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.673855 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.673865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.674482 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.674487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.674635 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.675076 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.675091 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.677621 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 02:53:11.677991 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.671020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:38615 (local address 127.8.57.62:41517)
0929 02:53:11.671372 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.671375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.671386 (+ 11us) server_negotiation.cc:407] Connection header received
0929 02:53:11.671418 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.671421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.671469 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.671542 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.672169 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.672858 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.673972 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.674361 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.674458 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.677788 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:38615: BlockingRecv error: recv got EOF from 127.8.57.3:38615 (error 108)
Metrics: {"server-negotiator.queue_time_us":278}
W20260929 02:53:11.678113 17649 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:11 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:11.824697 17654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:11.816845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:55869)
0929 02:53:11.817324 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:11.817339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:11.817461 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:11.817760 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:11.817765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:11.818206 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:11.818397 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.818407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.819446 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.819451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.820440 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:11.820453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.821156 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.821163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:11.821309 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.821819 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:11.821836 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:11.824467 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":239,"threads_started":1}
W20260929 02:53:11.824887 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:11.817009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:55869 (local address 127.8.57.62:41517)
0929 02:53:11.817220 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:11.817226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:11.817413 (+ 187us) server_negotiation.cc:407] Connection header received
0929 02:53:11.817569 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:11.817574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:11.817635 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:11.817713 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:11.818537 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.819313 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.820573 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:11.820992 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:11.821095 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:11.824662 (+ 3567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:55869: BlockingRecv error: recv got EOF from 127.8.57.1:55869 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 02:53:12 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:12 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:53:12.253021 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:12.244602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54339 (local address 127.8.57.62:41517)
0929 02:53:12.244794 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:12.244800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:12.245124 (+ 324us) server_negotiation.cc:407] Connection header received
0929 02:53:12.245282 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:12.245288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:12.245354 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:12.245428 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:12.246283 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.246976 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.248253 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.248671 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.248774 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:12.252761 (+ 3987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:54339: BlockingRecv error: recv got EOF from 127.8.57.2:54339 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:53:12.253701 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:12.244569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:54339)
0929 02:53:12.245028 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:12.245043 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:12.245170 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:12.245470 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:12.245474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:12.245936 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:12.246143 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:12.246153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.247109 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.247115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:12.248115 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:12.248127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.248833 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.248839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:12.248980 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:12.249524 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:12.249541 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:12.253419 (+ 3878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":242,"threads_started":1}
Sep 29 02:53:12 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:12 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:12.686065 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:12.678623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:56341)
0929 02:53:12.679046 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:12.679062 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:12.679174 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:12.679433 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:12.679436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:12.679865 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:12.680086 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:12.680096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.680999 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.681002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:12.681944 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:12.681955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.682553 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.682557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:12.682680 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:12.683204 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:12.683222 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:12.685833 (+ 2611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":217,"threads_started":1}
W20260929 02:53:12.686264 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:12.678667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:56341 (local address 127.8.57.62:41517)
0929 02:53:12.678853 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:12.678860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:12.679129 (+ 269us) server_negotiation.cc:407] Connection header received
0929 02:53:12.679267 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:12.679271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:12.679331 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:12.679405 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:12.680184 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.680894 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.682041 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.682426 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.682509 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:12.686031 (+ 3522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:56341: BlockingRecv error: recv got EOF from 127.8.57.3:56341 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 02:53:12 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:12 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:12.833287 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:12.825578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:56003)
0929 02:53:12.826065 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:12.826082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:12.826195 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:12.826468 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:12.826471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:12.826853 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:12.826992 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:12.827000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.827946 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.827950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:12.828856 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:12.828868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.829469 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.829473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:12.829575 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:12.830106 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:12.830123 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:12.833058 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":251,"threads_started":1}
W20260929 02:53:12.833427 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:12.825649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:56003 (local address 127.8.57.62:41517)
0929 02:53:12.825866 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:12.825872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:12.826154 (+ 282us) server_negotiation.cc:407] Connection header received
0929 02:53:12.826289 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:12.826293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:12.826353 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:12.826427 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:12.827095 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.827805 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.828966 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:12.829357 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:12.829448 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:12.833225 (+ 3777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:56003: BlockingRecv error: recv got EOF from 127.8.57.1:56003 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 02:53:13 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:13 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:13.265573 17658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:13.256067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:34963)
0929 02:53:13.257066 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:13.257087 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:13.257283 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:13.257925 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:13.257930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:13.258454 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:13.258666 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:13.258680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.259709 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.259714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:13.260676 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:13.260688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.261399 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.261405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:13.261557 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:13.262129 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:13.262146 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:13.265270 (+ 3124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":732,"threads_started":1}
W20260929 02:53:13.265813 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:13.255467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:34963 (local address 127.8.57.62:41517)
0929 02:53:13.255854 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:13.255861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:13.257165 (+ 1304us) server_negotiation.cc:407] Connection header received
0929 02:53:13.257424 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:13.257430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:13.257508 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:13.257586 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:13.258831 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.259559 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.260825 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.261248 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.261359 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:13.265541 (+ 4182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:34963: BlockingRecv error: recv got EOF from 127.8.57.2:34963 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 29 02:53:13 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:13 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:13.695392 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:13.686961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:48405)
0929 02:53:13.687473 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:13.687489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:13.687629 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:13.687994 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:13.687997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:13.688386 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:13.688556 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:13.688569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.689651 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.689655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:13.690605 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:13.690618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.691338 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.691343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:13.691456 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:13.692060 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:13.692079 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:13.695073 (+ 2994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":266,"threads_started":1}
W20260929 02:53:13.695572 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:13.687150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:48405 (local address 127.8.57.62:41517)
0929 02:53:13.687324 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:13.687328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:13.687558 (+ 230us) server_negotiation.cc:407] Connection header received
0929 02:53:13.687772 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:13.687776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:13.687839 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:13.687955 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:13.688710 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.689505 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.690755 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.691195 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.691297 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:13.695301 (+ 4004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:48405: BlockingRecv error: recv got EOF from 127.8.57.3:48405 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 29 02:53:13 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:13 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:13.841759 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:13.834203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:43031)
0929 02:53:13.834623 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:13.834639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:13.834755 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:13.835018 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:13.835021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:13.835398 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:13.835572 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:13.835580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.836591 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.836594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:13.837467 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:13.837479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.838062 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.838066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:13.838206 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:13.838759 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:13.838776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:13.841502 (+ 2726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":220,"threads_started":1}
W20260929 02:53:13.841917 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:13.834288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:43031 (local address 127.8.57.62:41517)
0929 02:53:13.834507 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:13.834513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:13.834717 (+ 204us) server_negotiation.cc:407] Connection header received
0929 02:53:13.834853 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:13.834856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:13.834918 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:13.834987 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:13.835687 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.836472 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.837574 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:13.837961 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:13.838045 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:13.841711 (+ 3666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:43031: BlockingRecv error: recv got EOF from 127.8.57.1:43031 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 29 02:53:14 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:14 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:14.275427 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:14.266660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:54789)
0929 02:53:14.267163 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:14.267180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:14.267304 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:14.267596 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:14.267600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:14.267988 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:14.268186 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:14.268196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.269448 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.269453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:14.270683 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:14.270698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.271360 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.271366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:14.271556 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:14.272266 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:14.272284 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:14.275083 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":257,"threads_started":1}
W20260929 02:53:14.275611 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:14.266767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54789 (local address 127.8.57.62:41517)
0929 02:53:14.266947 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:14.266951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:14.267289 (+ 338us) server_negotiation.cc:407] Connection header received
0929 02:53:14.267401 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:14.267406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:14.267473 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:14.267569 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:14.268342 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.269302 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.270802 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.271202 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.271315 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:14.275381 (+ 4066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:54789: BlockingRecv error: recv got EOF from 127.8.57.2:54789 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 29 02:53:14 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:14 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:14.705250 17662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:14.696458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:33791)
0929 02:53:14.696926 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:14.696944 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:14.697074 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:14.697447 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:14.697451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:14.697785 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:14.697951 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:14.697960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.698986 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.698990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:14.699955 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:14.699971 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.700733 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.700739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:14.700924 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:14.701631 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:14.701655 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:14.704960 (+ 3305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":252,"threads_started":1}
W20260929 02:53:14.706238 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:14.696557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:33791 (local address 127.8.57.62:41517)
0929 02:53:14.696711 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:14.696715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:14.697011 (+ 296us) server_negotiation.cc:407] Connection header received
0929 02:53:14.697229 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:14.697234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:14.697304 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:14.697397 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:14.698124 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.698845 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.700108 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.700584 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.700713 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:14.706004 (+ 5291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:33791: BlockingRecv error: recv got EOF from 127.8.57.3:33791 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 02:53:14 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:14 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:14.850909 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:14.842679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:57249)
0929 02:53:14.843154 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:14.843177 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:14.843294 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:14.843570 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:14.843574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:14.843984 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:14.844147 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:14.844156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.845163 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.845167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:14.846243 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:14.846258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.846902 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.846906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:14.847030 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:14.847619 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:14.847637 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:14.850662 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":251,"threads_started":1}
W20260929 02:53:14.851212 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:14.842777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:57249 (local address 127.8.57.62:41517)
0929 02:53:14.842957 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:14.842961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:14.843253 (+ 292us) server_negotiation.cc:407] Connection header received
0929 02:53:14.843388 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:14.843392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:14.843455 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:14.843538 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:14.844299 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.845041 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.846372 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:14.846791 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:14.846889 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:14.850946 (+ 4057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:57249: BlockingRecv error: recv got EOF from 127.8.57.1:57249 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 02:53:15 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:15 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:15.284901 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:15.276364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:50025)
0929 02:53:15.276825 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:15.276845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:15.276969 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:15.277320 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:15.277324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:15.277662 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:15.277805 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:15.277813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.278804 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.278808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:15.279764 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:15.279776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.280563 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.280568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:15.280705 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:15.281418 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:15.281436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:15.284619 (+ 3183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":248,"threads_started":1}
W20260929 02:53:15.285038 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:15.276422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:50025 (local address 127.8.57.62:41517)
0929 02:53:15.276584 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:15.276588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:15.276909 (+ 321us) server_negotiation.cc:407] Connection header received
0929 02:53:15.277132 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:15.277136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:15.277202 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:15.277290 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:15.277938 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.278687 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.279937 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.280415 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.280528 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:15.284792 (+ 4264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:50025: BlockingRecv error: recv got EOF from 127.8.57.2:50025 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 02:53:15 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:15 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:15.714509 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:15.706299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:53255)
0929 02:53:15.706803 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:15.706820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:15.706942 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:15.707291 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:15.707296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:15.707751 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:15.707975 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:15.707989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.709143 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.709148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:15.710099 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:15.710110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.710825 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.710831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:15.710995 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:15.711532 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:15.711549 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:15.714161 (+ 2612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":275,"threads_started":1}
W20260929 02:53:15.714636 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:15.706430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:53255 (local address 127.8.57.62:41517)
0929 02:53:15.706586 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:15.706591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:15.706892 (+ 301us) server_negotiation.cc:407] Connection header received
0929 02:53:15.707086 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:15.707090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:15.707161 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:15.707244 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:15.708100 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.709009 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.710230 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.710684 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.710789 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:15.714409 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:53255: BlockingRecv error: recv got EOF from 127.8.57.3:53255 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 02:53:15 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:15 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:15.859534 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:15.851918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:36507)
0929 02:53:15.852572 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:15.852588 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:15.852694 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:15.852959 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:15.852962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:15.853348 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:15.853507 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:15.853515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.854420 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.854424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:15.855330 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:15.855341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.855978 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.855981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:15.856103 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:15.856643 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:15.856661 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:15.859284 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":408,"threads_started":1}
W20260929 02:53:15.859644 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:15.851983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:36507 (local address 127.8.57.62:41517)
0929 02:53:15.852197 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:15.852201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:15.852664 (+ 463us) server_negotiation.cc:407] Connection header received
0929 02:53:15.852791 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:15.852794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:15.852854 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:15.852930 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:15.853618 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.854321 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.855434 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:15.855823 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:15.855962 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:15.859468 (+ 3506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:36507: BlockingRecv error: recv got EOF from 127.8.57.1:36507 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:53:16 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:16 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:16.298672 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:16.285842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:52299)
0929 02:53:16.286357 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:16.286374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:16.286486 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:16.286842 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:16.286848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:16.287405 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:16.287620 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:16.287632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.288913 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.288917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:16.289926 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:16.289941 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.290712 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.290716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:16.290849 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:16.295407 (+ 4558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:16.295428 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:16.298418 (+ 2990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":266,"threads_started":1}
W20260929 02:53:16.298970 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:16.286018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:52299 (local address 127.8.57.62:41517)
0929 02:53:16.286261 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:16.286267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:16.286453 (+ 186us) server_negotiation.cc:407] Connection header received
0929 02:53:16.286597 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:16.286602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:16.286681 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:16.286782 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:16.287766 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.288773 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.290108 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.290590 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.290691 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:16.298707 (+ 8016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:52299: BlockingRecv error: recv got EOF from 127.8.57.2:52299 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 02:53:16 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:16 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:16.723552 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:16.715462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:58267)
0929 02:53:16.716000 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:16.716016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:16.716159 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:16.716472 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:16.716476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:16.716920 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:16.717081 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:16.717091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.718124 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.718127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:16.719012 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:16.719025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.719692 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.719698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:16.719863 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:16.720442 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:16.720461 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:16.723221 (+ 2760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":255,"threads_started":1}
W20260929 02:53:16.723671 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:16.715607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:58267 (local address 127.8.57.62:41517)
0929 02:53:16.715836 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:16.715842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:16.716113 (+ 271us) server_negotiation.cc:407] Connection header received
0929 02:53:16.716282 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:16.716285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:16.716346 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:16.716424 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:16.717221 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.717961 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.719146 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.719556 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.719661 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:16.723452 (+ 3791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:58267: BlockingRecv error: recv got EOF from 127.8.57.3:58267 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 29 02:53:16 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:16 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:16.868211 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:16.860420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:60649)
0929 02:53:16.860901 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:16.860916 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:16.861044 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:16.861363 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:16.861368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:16.861758 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:16.861926 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:16.861935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.862930 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.862934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:16.863825 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:16.863837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.864585 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.864589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:16.864748 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:16.865361 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:16.865379 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:16.867986 (+ 2607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":249,"threads_started":1}
W20260929 02:53:16.868384 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:16.860488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:60649 (local address 127.8.57.62:41517)
0929 02:53:16.860659 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:16.860663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:16.860995 (+ 332us) server_negotiation.cc:407] Connection header received
0929 02:53:16.861153 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:16.861159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:16.861228 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:16.861305 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:16.862064 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.862794 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.863984 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:16.864452 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:16.864557 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:16.868174 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:60649: BlockingRecv error: recv got EOF from 127.8.57.1:60649 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 02:53:17 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:17 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:17.307745 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:17.299647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:42565)
0929 02:53:17.300391 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:17.300407 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:17.300523 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:17.300869 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:17.300874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:17.301349 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:17.301500 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:17.301509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.302449 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.302453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:17.303344 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:17.303355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.304040 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.304047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:17.304209 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:17.304766 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:17.304784 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:17.307512 (+ 2728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":514,"threads_started":1}
W20260929 02:53:17.308012 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:17.299902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:42565 (local address 127.8.57.62:41517)
0929 02:53:17.300086 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:17.300093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:17.300495 (+ 402us) server_negotiation.cc:407] Connection header received
0929 02:53:17.300644 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:17.300648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:17.300724 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:17.300822 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:17.301641 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.302347 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.303463 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.303860 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.304000 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:17.307752 (+ 3752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:42565: BlockingRecv error: recv got EOF from 127.8.57.2:42565 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 29 02:53:17 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:17 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:17.732687 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:17.724589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:50213)
0929 02:53:17.725106 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:17.725122 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:17.725248 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:17.725565 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:17.725570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:17.725963 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:17.726137 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:17.726148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.727140 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.727146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:17.728073 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:17.728086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.728877 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.728883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:17.729063 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:17.729666 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:17.729689 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:17.732372 (+ 2683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":280,"threads_started":1}
W20260929 02:53:17.732834 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:17.724724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:50213 (local address 127.8.57.62:41517)
0929 02:53:17.724933 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:17.724938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:17.725200 (+ 262us) server_negotiation.cc:407] Connection header received
0929 02:53:17.725372 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:17.725378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:17.725439 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:17.725516 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:17.726296 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.727015 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.728208 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.728728 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.728839 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:17.732611 (+ 3772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:50213: BlockingRecv error: recv got EOF from 127.8.57.3:50213 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 02:53:17 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:17 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:17.876650 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:17.869037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:48353)
0929 02:53:17.869635 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:17.869654 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:17.869778 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:17.870093 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:17.870096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:17.870583 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:17.870747 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:17.870756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.871683 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.871686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:17.872614 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:17.872625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.873269 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.873273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:17.873376 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:17.873863 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:17.873879 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:17.876416 (+ 2537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":219,"threads_started":1}
W20260929 02:53:17.876770 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:17.869109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:48353 (local address 127.8.57.62:41517)
0929 02:53:17.869296 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:17.869302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:17.869728 (+ 426us) server_negotiation.cc:407] Connection header received
0929 02:53:17.869880 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:17.869885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:17.869963 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:17.870048 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:17.870867 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.871571 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.872737 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:17.873157 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:17.873262 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:17.876591 (+ 3329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:48353: BlockingRecv error: recv got EOF from 127.8.57.1:48353 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 02:53:18 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:18 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:18.317186 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:18.308646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:49301)
0929 02:53:18.309221 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:18.309237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:18.309350 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:18.309623 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:18.309627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:18.310128 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:18.310307 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:18.310316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.311405 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.311409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:18.312351 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:18.312363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.313036 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.313042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:18.313185 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:18.313834 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:18.313852 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:18.316863 (+ 3011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":257,"threads_started":1}
W20260929 02:53:18.317301 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:18.308719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:49301 (local address 127.8.57.62:41517)
0929 02:53:18.308919 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:18.308923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:18.309301 (+ 378us) server_negotiation.cc:407] Connection header received
0929 02:53:18.309427 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:18.309431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:18.309501 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:18.309582 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:18.310447 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.311263 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.312478 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.312892 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.312998 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:18.317065 (+ 4067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:49301: BlockingRecv error: recv got EOF from 127.8.57.2:49301 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 29 02:53:18 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:18 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:18.741670 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:18.733638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:35203)
0929 02:53:18.734300 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:18.734316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:18.734443 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:18.734750 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:18.734755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:18.735152 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:18.735314 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:18.735324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.736362 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.736367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:18.737254 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:18.737266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.737933 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.737939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:18.738090 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:18.738607 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:18.738625 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:18.741320 (+ 2695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":281,"threads_started":1}
W20260929 02:53:18.741750 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:18.733701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:35203 (local address 127.8.57.62:41517)
0929 02:53:18.733891 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:18.733898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:18.734394 (+ 496us) server_negotiation.cc:407] Connection header received
0929 02:53:18.734562 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:18.734567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:18.734627 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:18.734702 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:18.735456 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.736242 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.737385 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.737794 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.737903 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:18.741534 (+ 3631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:35203: BlockingRecv error: recv got EOF from 127.8.57.3:35203 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:53:18 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:18 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:18.885272 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:18.877474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:54207)
0929 02:53:18.877954 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:18.877969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:18.878078 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:18.878379 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:18.878382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:18.878759 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:18.878908 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:18.878916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.879822 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.879826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:18.880756 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:18.880766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.881490 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.881494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:18.881605 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:18.882177 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:18.882195 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:18.885036 (+ 2841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":223,"threads_started":1}
W20260929 02:53:18.885433 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:18.877624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:54207 (local address 127.8.57.62:41517)
0929 02:53:18.877805 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:18.877809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:18.878031 (+ 222us) server_negotiation.cc:407] Connection header received
0929 02:53:18.878208 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:18.878212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:18.878272 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:18.878348 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:18.879012 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.879715 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.880893 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:18.881343 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:18.881437 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:18.885230 (+ 3793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:54207: BlockingRecv error: recv got EOF from 127.8.57.1:54207 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 02:53:19 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:19 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:19.325937 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:19.318109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:41267)
0929 02:53:19.318572 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:19.318592 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:19.318729 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:19.319075 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:19.319079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:19.319439 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:19.319592 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:19.319600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.320749 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.320753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:19.321645 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:19.321655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.322364 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.322368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:19.322472 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:19.322977 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:19.322993 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:19.325720 (+ 2727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
W20260929 02:53:19.326090 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:19.318186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41267 (local address 127.8.57.62:41517)
0929 02:53:19.318364 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:19.318370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:19.318665 (+ 295us) server_negotiation.cc:407] Connection header received
0929 02:53:19.318858 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:19.318863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:19.318938 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:19.319032 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:19.319732 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.320632 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.321779 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.322214 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.322322 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:19.325906 (+ 3584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:41267: BlockingRecv error: recv got EOF from 127.8.57.2:41267 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 02:53:19 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:19 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:19.750545 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:19.742622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:36609)
0929 02:53:19.743058 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:19.743074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:19.743191 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:19.743505 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:19.743511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:19.743917 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:19.744072 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:19.744081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.745077 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.745083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:19.745997 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:19.746009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.746727 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.746733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:19.746882 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:19.747532 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:19.747555 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:19.750330 (+ 2775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":208,"threads_started":1}
W20260929 02:53:19.750753 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:19.742701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:36609 (local address 127.8.57.62:41517)
0929 02:53:19.742910 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:19.742916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:19.743181 (+ 265us) server_negotiation.cc:407] Connection header received
0929 02:53:19.743283 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:19.743286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:19.743349 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:19.743438 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:19.744225 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.744940 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.746144 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.746553 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.746654 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:19.750514 (+ 3860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:36609: BlockingRecv error: recv got EOF from 127.8.57.3:36609 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 02:53:19 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:19 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:19.894357 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:19.886298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:43631)
0929 02:53:19.886824 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:19.886840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:19.886946 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:19.887237 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:19.887241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:19.887629 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:19.887827 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:19.887838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.888852 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.888856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:19.889835 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:19.889851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.890634 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.890639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:19.890796 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:19.891375 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:19.891394 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:19.894099 (+ 2705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":246,"threads_started":1}
W20260929 02:53:19.894527 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:19.886315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:43631 (local address 127.8.57.62:41517)
0929 02:53:19.886512 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:19.886518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:19.886912 (+ 394us) server_negotiation.cc:407] Connection header received
0929 02:53:19.887037 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:19.887040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:19.887100 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:19.887173 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:19.887980 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.888735 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.889998 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:19.890461 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:19.890561 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:19.894322 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:43631: BlockingRecv error: recv got EOF from 127.8.57.1:43631 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:53:20 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:20 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:20.334625 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:20.326797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:57625)
0929 02:53:20.327281 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:20.327297 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:20.327406 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:20.327678 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:20.327681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:20.328087 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:20.328265 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:20.328274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.329400 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.329404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:20.330604 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:20.330616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.331255 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.331259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:20.331382 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:20.331905 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:20.331923 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:20.334398 (+ 2475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":235,"threads_started":1}
W20260929 02:53:20.334795 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:20.326888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:57625 (local address 127.8.57.62:41517)
0929 02:53:20.327049 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:20.327054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:20.327394 (+ 340us) server_negotiation.cc:407] Connection header received
0929 02:53:20.327498 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:20.327502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:20.327567 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:20.327646 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:20.328388 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.329281 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.330736 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.331125 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.331224 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:20.334579 (+ 3355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:57625: BlockingRecv error: recv got EOF from 127.8.57.2:57625 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 02:53:20 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:20 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:20.759111 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:20.751327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:40893)
0929 02:53:20.751791 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:20.751807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:20.751950 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:20.752263 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:20.752267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:20.752643 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:20.752803 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:20.752813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.753753 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.753759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:20.754686 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:20.754699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.755358 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.755364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:20.755528 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:20.756070 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:20.756087 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:20.758781 (+ 2694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":242,"threads_started":1}
W20260929 02:53:20.759220 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:20.751410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:40893 (local address 127.8.57.62:41517)
0929 02:53:20.751604 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:20.751610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:20.751898 (+ 288us) server_negotiation.cc:407] Connection header received
0929 02:53:20.752063 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:20.752069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:20.752135 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:20.752214 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:20.752930 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.753623 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.754808 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.755225 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.755325 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:20.759024 (+ 3699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:40893: BlockingRecv error: recv got EOF from 127.8.57.3:40893 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 02:53:20 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:20 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:20.902982 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:20.895271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:49341)
0929 02:53:20.895690 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:20.895706 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:20.895809 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:20.896079 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:20.896083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:20.896450 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:20.896584 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:20.896592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.897490 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.897493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:20.898414 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:20.898425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.899078 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.899082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:20.899245 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:20.899937 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:20.899962 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:20.902765 (+ 2803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":234,"threads_started":1}
W20260929 02:53:20.903131 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:20.895308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:49341 (local address 127.8.57.62:41517)
0929 02:53:20.895505 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:20.895513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:20.895773 (+ 260us) server_negotiation.cc:407] Connection header received
0929 02:53:20.895917 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:20.895921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:20.895979 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:20.896052 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:20.896706 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.897400 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.898515 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:20.898956 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:20.899038 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:20.902931 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:49341: BlockingRecv error: recv got EOF from 127.8.57.1:49341 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 02:53:21 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:21 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:21.347127 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:21.335471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:60301)
0929 02:53:21.336000 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:21.336016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:21.336140 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:21.336437 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:21.336441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:21.336820 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:21.336958 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:21.336967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.337917 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.337923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:21.338908 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:21.338921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.339576 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.339582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:21.339738 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:21.344200 (+ 4462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:21.344217 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:21.346914 (+ 2697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":283,"threads_started":1}
W20260929 02:53:21.347425 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:21.335542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60301 (local address 127.8.57.62:41517)
0929 02:53:21.335719 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:21.335726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:21.336094 (+ 368us) server_negotiation.cc:407] Connection header received
0929 02:53:21.336256 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:21.336260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:21.336317 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:21.336394 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:21.337093 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.337795 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.339047 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.339441 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.339543 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:21.347094 (+ 7551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:60301: BlockingRecv error: recv got EOF from 127.8.57.2:60301 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 02:53:21 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:21 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:21.767980 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:21.759978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:45259)
0929 02:53:21.760430 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:21.760445 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:21.760558 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:21.760877 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:21.760882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:21.761289 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:21.761432 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:21.761441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.762428 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.762433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:21.763343 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:21.763355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.764079 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.764084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:21.764240 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:21.764839 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:21.764863 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:21.767726 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":233,"threads_started":1}
W20260929 02:53:21.768118 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:21.760156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:45259 (local address 127.8.57.62:41517)
0929 02:53:21.760382 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:21.760387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:21.760507 (+ 120us) server_negotiation.cc:407] Connection header received
0929 02:53:21.760683 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:21.760688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:21.760748 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:21.760824 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:21.761560 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.762299 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.763478 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.763946 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.764051 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:21.767932 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:45259: BlockingRecv error: recv got EOF from 127.8.57.3:45259 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Sep 29 02:53:21 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:21 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:21.911741 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:21.903819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:53321)
0929 02:53:21.904489 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:21.904504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:21.904611 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:21.904934 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:21.904939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:21.905401 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:21.905569 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:21.905578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.906533 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.906536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:21.907422 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:21.907432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.908235 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.908241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:21.908360 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:21.908859 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:21.908875 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:21.911513 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":256,"threads_started":1}
W20260929 02:53:21.911908 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:21.903931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:53321 (local address 127.8.57.62:41517)
0929 02:53:21.904132 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:21.904138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:21.904582 (+ 444us) server_negotiation.cc:407] Connection header received
0929 02:53:21.904703 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:21.904706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:21.904793 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:21.904902 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:21.905689 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.906431 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.907547 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:21.908064 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:21.908180 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:21.911706 (+ 3526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:53321: BlockingRecv error: recv got EOF from 127.8.57.1:53321 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 02:53:22 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:22 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:22.355342 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:22.347979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:51607)
0929 02:53:22.348466 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:22.348481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:22.348600 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:22.348847 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:22.348850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:22.349230 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:22.349393 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:22.349401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.350356 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.350361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:22.351241 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:22.351252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.351836 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.351839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:22.351995 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:22.352537 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:22.352554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:22.355112 (+ 2558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":271,"threads_started":1}
W20260929 02:53:22.355489 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:22.348020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:51607 (local address 127.8.57.62:41517)
0929 02:53:22.348198 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:22.348203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:22.348543 (+ 340us) server_negotiation.cc:407] Connection header received
0929 02:53:22.348685 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:22.348688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:22.348745 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:22.348818 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:22.349495 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.350225 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.351341 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.351733 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.351821 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:22.355288 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:51607: BlockingRecv error: recv got EOF from 127.8.57.2:51607 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 02:53:22 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:22 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:22.776651 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:22.768999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:37873)
0929 02:53:22.769460 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:22.769476 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:22.769581 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:22.769841 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:22.769843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:22.770242 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:22.770422 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:22.770430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.771379 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.771382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:22.772346 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:22.772357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.773012 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.773022 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:22.773137 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:22.773682 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:22.773700 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:22.776318 (+ 2618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":258,"threads_started":1}
W20260929 02:53:22.776748 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:22.769044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:37873 (local address 127.8.57.62:41517)
0929 02:53:22.769225 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:22.769232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:22.769545 (+ 313us) server_negotiation.cc:407] Connection header received
0929 02:53:22.769674 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:22.769677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:22.769736 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:22.769807 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:22.770540 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.771280 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.772480 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.772878 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.772977 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:22.776509 (+ 3532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:37873: BlockingRecv error: recv got EOF from 127.8.57.3:37873 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:53:22 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:22 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:22.920169 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:22.912562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:55879)
0929 02:53:22.913097 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:22.913112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:22.913226 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:22.913472 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:22.913475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:22.913864 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:22.914015 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:22.914022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.915015 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.915020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:22.916080 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:22.916092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.916832 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.916836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:22.916957 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:22.917462 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:22.917479 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:22.919953 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":342,"threads_started":1}
W20260929 02:53:22.920359 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:22.912631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:55879 (local address 127.8.57.62:41517)
0929 02:53:22.912807 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:22.912811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:22.913205 (+ 394us) server_negotiation.cc:407] Connection header received
0929 02:53:22.913308 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:22.913311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:22.913371 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:22.913444 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:22.914123 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.914893 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.916189 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:22.916712 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:22.916801 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:22.920132 (+ 3331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:55879: BlockingRecv error: recv got EOF from 127.8.57.1:55879 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 02:53:23 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:23 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:23.364123 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:23.356091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:41571)
0929 02:53:23.356621 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:23.356639 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:23.356810 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:23.357118 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:23.357122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:23.357515 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:23.357695 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:23.357706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.358756 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.358762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:23.359660 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:23.359680 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.360398 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.360404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:23.360573 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:23.361123 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:23.361139 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:23.363763 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":251,"threads_started":1}
W20260929 02:53:23.364203 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:23.356177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41571 (local address 127.8.57.62:41517)
0929 02:53:23.356377 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:23.356383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:23.356747 (+ 364us) server_negotiation.cc:407] Connection header received
0929 02:53:23.356935 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:23.356940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:23.356998 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:23.357075 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:23.357823 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.358631 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.359791 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.360271 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.360377 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:23.363997 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:41571: BlockingRecv error: recv got EOF from 127.8.57.2:41571 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:53:23 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:23 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:23.785727 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:23.777558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:48881)
0929 02:53:23.777997 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:23.778013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:23.778129 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:23.778402 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:23.778405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:23.778770 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:23.778939 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:23.778949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.780387 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.780392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:23.781303 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:23.781315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.782173 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.782177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:23.782298 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:23.782819 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:23.782836 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:23.785509 (+ 2673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":238,"threads_started":1}
W20260929 02:53:23.785900 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:23.777598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:48881 (local address 127.8.57.62:41517)
0929 02:53:23.777803 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:23.777810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:23.778079 (+ 269us) server_negotiation.cc:407] Connection header received
0929 02:53:23.778237 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:23.778241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:23.778299 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:23.778367 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:23.779097 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.780251 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.781427 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.782045 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.782172 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:23.785699 (+ 3527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:48881: BlockingRecv error: recv got EOF from 127.8.57.3:48881 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:53:23 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:23 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:23.928900 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:23.921004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:47851)
0929 02:53:23.921476 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:23.921492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:23.921615 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:23.921924 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:23.921928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:23.922312 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:23.922463 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:23.922472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.923438 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.923443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:23.924358 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:23.924369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.925219 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.925224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:23.925415 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:23.926008 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:23.926030 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:23.928652 (+ 2622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
W20260929 02:53:23.928987 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:23.921101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:47851 (local address 127.8.57.62:41517)
0929 02:53:23.921315 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:23.921321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:23.921568 (+ 247us) server_negotiation.cc:407] Connection header received
0929 02:53:23.921735 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:23.921740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:23.921805 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:23.921882 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:23.922595 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.923309 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.924484 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:23.925086 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:23.925211 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:23.928809 (+ 3598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:47851: BlockingRecv error: recv got EOF from 127.8.57.1:47851 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 02:53:24 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:24 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:24.372570 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:24.365015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:55643)
0929 02:53:24.365490 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:24.365504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:24.365609 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:24.365880 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:24.365884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:24.366281 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:24.366434 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:24.366442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.367388 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.367394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:24.368453 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:24.368464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.369193 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.369197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:24.369314 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:24.369844 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:24.369861 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:24.372350 (+ 2489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
W20260929 02:53:24.372738 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:24.365131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:55643 (local address 127.8.57.62:41517)
0929 02:53:24.365283 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:24.365287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:24.365552 (+ 265us) server_negotiation.cc:407] Connection header received
0929 02:53:24.365716 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:24.365719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:24.365776 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:24.365851 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:24.366534 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.367277 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.368553 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.369044 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.369134 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:24.372527 (+ 3393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:55643: BlockingRecv error: recv got EOF from 127.8.57.2:55643 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 02:53:24 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:24 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:24.794538 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:24.786623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:38923)
0929 02:53:24.787131 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:24.787146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:24.787260 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:24.787604 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:24.787609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:24.788100 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:24.788306 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:24.788318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.789416 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.789420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:24.790322 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:24.790336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.790948 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.790952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:24.791061 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:24.791560 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:24.791576 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:24.794207 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
W20260929 02:53:24.794610 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:24.786709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:38923 (local address 127.8.57.62:41517)
0929 02:53:24.786860 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:24.786864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:24.787224 (+ 360us) server_negotiation.cc:407] Connection header received
0929 02:53:24.787375 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:24.787380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:24.787457 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:24.787577 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:24.788426 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.789304 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.790430 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.790846 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.790934 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:24.794411 (+ 3477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:38923: BlockingRecv error: recv got EOF from 127.8.57.3:38923 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 02:53:24 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:24 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:24.937466 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:24.929664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:37867)
0929 02:53:24.930185 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:24.930202 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:24.930341 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:24.930653 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:24.930658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:24.931004 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:24.931214 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:24.931224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.932262 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.932265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:24.933173 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:24.933210 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.933898 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.933903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:24.934027 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:24.934619 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:24.934638 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:24.937246 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":239,"threads_started":1}
W20260929 02:53:24.937604 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:24.929773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:37867 (local address 127.8.57.62:41517)
0929 02:53:24.930046 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:24.930053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:24.930286 (+ 233us) server_negotiation.cc:407] Connection header received
0929 02:53:24.930470 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:24.930474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:24.930533 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:24.930621 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:24.931351 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.932128 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.933340 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:24.933772 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:24.933863 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:24.937405 (+ 3542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:37867: BlockingRecv error: recv got EOF from 127.8.57.1:37867 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Sep 29 02:53:25 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:25 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:25.381249 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:25.373521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:60867)
0929 02:53:25.374018 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:25.374036 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:25.374161 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:25.374454 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:25.374459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:25.374835 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:25.374996 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:25.375005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.376018 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.376024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:25.376925 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:25.376936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.377617 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.377622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:25.377790 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:25.378331 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:25.378348 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:25.380920 (+ 2572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":268,"threads_started":1}
W20260929 02:53:25.381328 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:25.373567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60867 (local address 127.8.57.62:41517)
0929 02:53:25.373743 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:25.373749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:25.374111 (+ 362us) server_negotiation.cc:407] Connection header received
0929 02:53:25.374272 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:25.374277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:25.374338 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:25.374413 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:25.375135 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.375893 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.377053 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.377490 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.377592 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:25.381121 (+ 3529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:60867: BlockingRecv error: recv got EOF from 127.8.57.2:60867 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 29 02:53:25 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:25 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.3@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:25.803310 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:25.795426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.3:52665)
0929 02:53:25.795926 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:25.795942 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:25.796058 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:25.796347 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:25.796350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:25.796684 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:25.796857 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:25.796866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.797772 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.797776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:25.798775 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:25.798787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.799630 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.799635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:25.799809 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:25.800437 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:25.800453 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:25.803009 (+ 2556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":262,"threads_started":1}
W20260929 02:53:25.803489 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:25.795530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:52665 (local address 127.8.57.62:41517)
0929 02:53:25.795735 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:25.795742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:25.796011 (+ 269us) server_negotiation.cc:407] Connection header received
0929 02:53:25.796153 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:25.796157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:25.796242 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:25.796319 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:25.796964 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.797660 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.798914 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.799493 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.799593 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:25.803277 (+ 3684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.3:52665: BlockingRecv error: recv got EOF from 127.8.57.3:52665 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 02:53:25 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:25 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.1@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:25.946437 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:25.938553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.1:51859)
0929 02:53:25.939053 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:25.939072 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:25.939242 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:25.939592 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:25.939596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:25.940106 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:25.940368 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:25.940382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.941367 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.941370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:25.942303 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:25.942314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.942941 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.942945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:25.943048 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:25.943572 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:25.943589 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:25.946183 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
W20260929 02:53:25.946573 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:25.938659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:51859 (local address 127.8.57.62:41517)
0929 02:53:25.938864 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:25.938871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:25.939162 (+ 291us) server_negotiation.cc:407] Connection header received
0929 02:53:25.939377 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:25.939382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:25.939458 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:25.939538 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:25.940523 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.941247 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.942403 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:25.942820 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:25.942906 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:25.946375 (+ 3469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.1:51859: BlockingRecv error: recv got EOF from 127.8.57.1:51859 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 29 02:53:26 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:53:26 dist-test-slave-1dkt krb5kdc[16652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.57.2@KRBTEST.COM for oryx/127.8.57.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:53:26.394519 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:26.382337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41517 (local address 127.8.57.2:41213)
0929 02:53:26.382804 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:26.382820 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:26.382933 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:26.383318 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:26.383321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:26.383720 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:26.383935 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:26.383945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.385022 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.385026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:26.385969 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:26.385982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.386695 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.386700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:26.386826 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:26.391567 (+ 4741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:26.391587 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:26.394272 (+ 2685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.57.62:41517: Server oryx/127.8.57.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":244,"threads_started":1}
W20260929 02:53:26.394678 17388 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:26.382456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41213 (local address 127.8.57.62:41517)
0929 02:53:26.382668 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:26.382675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:26.382911 (+ 236us) server_negotiation.cc:407] Connection header received
0929 02:53:26.383071 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:26.383075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:26.383177 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:26.383281 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:26.384066 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.384867 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.386109 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.386549 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.386635 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:26.394455 (+ 7820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.57.2:41213: BlockingRecv error: recv got EOF from 127.8.57.2:41213 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:53:26.657492 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17239
I20260929 02:53:26.668615 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:26.658493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.8.57.1:38573)
0929 02:53:26.658967 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:26.658974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:26.658998 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:53:26.659055 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:26.659061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:26.659279 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:26.659422 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:26.660814 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.661790 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.662911 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.663441 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.663534 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:26.665760 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:26.665768 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:26.665781 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:26.665812 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:26.668159 (+ 2347us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.57.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:53:26.668398 (+ 239us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38684: Request ticket server kudu/127.8.57.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":214,"threads_started":1}
W20260929 02:53:26.668725 17699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38684: Request ticket server kudu/127.8.57.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:53:26.668757 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17239: Not authorized: Client connection negotiation failed: client connection to 127.8.57.1:38573: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.57.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:53:26.669076 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17239
I20260929 02:53:26.682134 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17390
I20260929 02:53:26.693398 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:26.682995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.8.57.2:40571)
0929 02:53:26.683519 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:26.683526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:26.683556 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:53:26.683613 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:26.683620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:26.683849 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:26.684065 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:26.685164 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.686234 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.687708 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.688305 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.688402 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:26.690724 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:26.690732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:26.690746 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:26.690780 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:26.692928 (+ 2148us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.57.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:53:26.693161 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50172: Request ticket server kudu/127.8.57.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":223,"threads_started":1}
W20260929 02:53:26.693485 17700 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50172: Request ticket server kudu/127.8.57.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:53:26.693555 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17390: Not authorized: Client connection negotiation failed: client connection to 127.8.57.2:40571: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.57.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:53:26.693691 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17390
I20260929 02:53:26.705865 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17521
I20260929 02:53:26.716674 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:26.706753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.8.57.3:41035)
0929 02:53:26.707304 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:26.707312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:26.707336 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:53:26.707392 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:26.707398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:26.707604 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:26.707749 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:26.708976 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.710054 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.711121 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:26.711621 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:26.711716 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:26.713880 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:26.713888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:26.713900 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:26.713939 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:26.716193 (+ 2254us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.57.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:53:26.716416 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57468: Request ticket server kudu/127.8.57.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":282,"threads_started":1}
W20260929 02:53:26.716769 17701 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57468: Request ticket server kudu/127.8.57.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:53:26.716729 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17521: Not authorized: Client connection negotiation failed: client connection to 127.8.57.3:41035: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.57.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:53:26.716892 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17521
I20260929 02:53:26.729293 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17170
I20260929 02:53:26.730408 17232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:26.843534 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17170
2026-09-29T02:53:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20641 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T02:53:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:26Z Disabled control of system clock
I20260929 02:53:26.909044 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37917
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 02:53:27.218302 17706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:27.218573 17706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:27.218694 17706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:27.225729 17706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:27.225827 17706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:27.225888 17706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:27.225963 17706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:27.226020 17706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:27.234992 17706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:27.236232 17706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:27.237905 17706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:27.249032 17712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:27.249068 17711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:27.249035 17714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:27.249265 17706 server_base.cc:1034] running on GCE node
I20260929 02:53:27.250319 17706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:27.251580 17706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:27.252791 17706 hybrid_clock.cc:648] HybridClock initialized: now 1790650407252741 us; error 35 us; skew 500 ppm
I20260929 02:53:27.253098 17706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:27.255093 17706 webserver.cc:460] Webserver started at http://127.8.57.62:45733/ using document root <none> and password file <none>
I20260929 02:53:27.255748 17706 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:27.255839 17706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:27.256179 17706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:27.258834 17706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "97e46f51d17643fa9d052c0965b7be6b"
format_stamp: "Formatted at 2026-09-29 02:53:27 on dist-test-slave-1dkt"
server_key: "b7d64542a2dafae7bc6ad903a10dbc1d"
server_key_iv: "4e9d84c97f4d79409c3fef4325de74ce"
server_key_version: "encryptionkey@0"
I20260929 02:53:27.259548 17706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "97e46f51d17643fa9d052c0965b7be6b"
format_stamp: "Formatted at 2026-09-29 02:53:27 on dist-test-slave-1dkt"
server_key: "b7d64542a2dafae7bc6ad903a10dbc1d"
server_key_iv: "4e9d84c97f4d79409c3fef4325de74ce"
server_key_version: "encryptionkey@0"
I20260929 02:53:27.264364 17706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260929 02:53:27.267611 17720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:27.268923 17706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:27.269080 17706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "97e46f51d17643fa9d052c0965b7be6b"
format_stamp: "Formatted at 2026-09-29 02:53:27 on dist-test-slave-1dkt"
server_key: "b7d64542a2dafae7bc6ad903a10dbc1d"
server_key_iv: "4e9d84c97f4d79409c3fef4325de74ce"
server_key_version: "encryptionkey@0"
I20260929 02:53:27.269279 17706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:27.285722 17706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:27.286550 17706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:27.286806 17706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:27.303097 17706 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:37917
I20260929 02:53:27.303128 17771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:37917 every 8 connection(s)
I20260929 02:53:27.304396 17706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:53:27.306412 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17706
I20260929 02:53:27.306591 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:53:27.307375 8420 external_mini_cluster.cc:1428] Setting key 9dfc6f6888f0d0cd9640f3298b279637
I20260929 02:53:27.309015 17772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:27.321945 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b: Bootstrap starting.
I20260929 02:53:27.324085 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:27.309567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.8.57.62:37917)
0929 02:53:27.310621 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:27.310636 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:27.312462 (+ 1826us) server_negotiation.cc:407] Connection header received
0929 02:53:27.314725 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:27.314751 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:27.315342 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:27.315775 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:27.316293 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:27.318071 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:27.319331 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:27.320361 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:27.320522 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:27.321505 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:27.321514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:27.321531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:27.322172 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:27.322277 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:27.322300 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:27.322403 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:27.322554 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 02:53:27.322847 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":315,"threads_started":1}
I20260929 02:53:27.325678 17772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:27.326704 17772 log.cc:824] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:27.329167 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b: No bootstrap required, opened a new log
I20260929 02:53:27.336213 17772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e46f51d17643fa9d052c0965b7be6b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37917 } }
I20260929 02:53:27.336525 17772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:27.336594 17772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e46f51d17643fa9d052c0965b7be6b, State: Initialized, Role: FOLLOWER
I20260929 02:53:27.337091 17772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e46f51d17643fa9d052c0965b7be6b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37917 } }
I20260929 02:53:27.337284 17772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:53:27.337383 17772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:53:27.337500 17772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:27.339439 17772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e46f51d17643fa9d052c0965b7be6b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37917 } }
I20260929 02:53:27.339864 17772 leader_election.cc:302] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97e46f51d17643fa9d052c0965b7be6b; no voters:
I20260929 02:53:27.340386 17772 leader_election.cc:288] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:53:27.340536 17777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:27.341190 17777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [term 1 LEADER]: Becoming Leader. State: Replica: 97e46f51d17643fa9d052c0965b7be6b, State: Running, Role: LEADER
I20260929 02:53:27.341653 17777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e46f51d17643fa9d052c0965b7be6b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37917 } }
I20260929 02:53:27.342227 17772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:53:27.344292 17779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e46f51d17643fa9d052c0965b7be6b. Latest consensus state: current_term: 1 leader_uuid: "97e46f51d17643fa9d052c0965b7be6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e46f51d17643fa9d052c0965b7be6b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37917 } } }
I20260929 02:53:27.344692 17779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:27.344571 17778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97e46f51d17643fa9d052c0965b7be6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e46f51d17643fa9d052c0965b7be6b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37917 } } }
I20260929 02:53:27.344901 17778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b [sys.catalog]: This master's current role is: LEADER
W20260929 02:53:27.352119 17793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:53:27.352237 17793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:53:27.352362 17785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:53:27.354251 17785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:53:27.362440 17785 catalog_manager.cc:1347] Generated new cluster ID: a169dc52dd4d419fb404b07226e8e8af
I20260929 02:53:27.362532 17785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:53:27.394833 17785 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:53:27.396133 17785 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:53:27.402511 17785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97e46f51d17643fa9d052c0965b7be6b: Generated new TSK 0
I20260929 02:53:27.403287 17785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:53:27.418212 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37917
--builtin_ntp_servers=127.8.57.20:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 02:53:27.727665 17796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:27.727957 17796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:27.728084 17796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:27.735302 17796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:27.735435 17796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:27.735580 17796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:53:27.744735 17796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:27.746125 17796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:27.747915 17796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:27.760083 17801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:27.760203 17802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:27.760203 17804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:27.760898 17796 server_base.cc:1034] running on GCE node
I20260929 02:53:27.761409 17796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:27.762254 17796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:27.763437 17796 hybrid_clock.cc:648] HybridClock initialized: now 1790650407763386 us; error 39 us; skew 500 ppm
I20260929 02:53:27.763734 17796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:27.765903 17796 webserver.cc:460] Webserver started at http://127.8.57.1:44315/ using document root <none> and password file <none>
I20260929 02:53:27.766554 17796 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:27.766652 17796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:27.766953 17796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:27.769721 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "1983283bbe7543acb77c5dbf0f373332"
format_stamp: "Formatted at 2026-09-29 02:53:27 on dist-test-slave-1dkt"
server_key: "aca83d3ea1d999847b0a5b4ab1d58fb7"
server_key_iv: "c8b9f6ffa9a8ca931fea895e77a0f87d"
server_key_version: "encryptionkey@0"
I20260929 02:53:27.770464 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "1983283bbe7543acb77c5dbf0f373332"
format_stamp: "Formatted at 2026-09-29 02:53:27 on dist-test-slave-1dkt"
server_key: "aca83d3ea1d999847b0a5b4ab1d58fb7"
server_key_iv: "c8b9f6ffa9a8ca931fea895e77a0f87d"
server_key_version: "encryptionkey@0"
I20260929 02:53:27.775347 17796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:53:27.778633 17810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:27.779990 17796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:53:27.780138 17796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "1983283bbe7543acb77c5dbf0f373332"
format_stamp: "Formatted at 2026-09-29 02:53:27 on dist-test-slave-1dkt"
server_key: "aca83d3ea1d999847b0a5b4ab1d58fb7"
server_key_iv: "c8b9f6ffa9a8ca931fea895e77a0f87d"
server_key_version: "encryptionkey@0"
I20260929 02:53:27.780336 17796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:27.798028 17796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:27.798866 17796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:27.799108 17796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:27.800047 17796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:27.801424 17796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:27.801522 17796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:27.801625 17796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:27.801698 17796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:27.829680 17796 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:40819
I20260929 02:53:27.829733 17922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:40819 every 8 connection(s)
I20260929 02:53:27.830893 17796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:53:27.837182 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17796
I20260929 02:53:27.837353 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:53:27.837990 8420 external_mini_cluster.cc:1428] Setting key 868217148bf3b3ae512071609bffa59d
I20260929 02:53:27.840909 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37917
--builtin_ntp_servers=127.8.57.20:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 02:53:27.843565 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:27.833316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:60497 (local address 127.8.57.62:37917)
0929 02:53:27.833764 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:27.833771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:27.834836 (+ 1065us) server_negotiation.cc:407] Connection header received
0929 02:53:27.836618 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:27.836623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:27.836688 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:27.836799 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:27.837850 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:27.838822 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:27.840835 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:27.841404 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:27.841525 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:27.842783 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:27.842789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:27.842793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:27.842812 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:27.842862 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:27.842876 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:27.842975 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:27.843144 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 02:53:27.843248 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":203,"threads_started":1}
I20260929 02:53:27.844692 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:27.833851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37917 (local address 127.8.57.1:60497)
0929 02:53:27.834688 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:27.834727 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:27.836334 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:27.836959 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:27.836966 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:27.837055 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:27.837688 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:27.837705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:27.838971 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:27.838975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:27.840689 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:27.840705 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:27.841624 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:27.841629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:27.841832 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:27.842493 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:53:27.842495 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:27.842537 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:53:27.842616 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:27.842957 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:27.842966 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:27.842974 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:53:27.843127 (+ 153us) client_negotiation.cc:240] Negotiation successful
0929 02:53:27.843457 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":259,"threads_started":1}
I20260929 02:53:27.846334 17923 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37917
I20260929 02:53:27.846618 17923 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:27.847440 17923 heartbeater.cc:511] Master 127.8.57.62:37917 requested a full tablet report, sending...
I20260929 02:53:27.849658 17737 ts_manager.cc:194] Registered new tserver with Master: 1983283bbe7543acb77c5dbf0f373332 (127.8.57.1:40819)
I20260929 02:53:27.851601 17737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.1:60497
W20260929 02:53:28.159377 17927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:28.159641 17927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:28.159723 17927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:28.166889 17927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:28.166990 17927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:28.167095 17927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:53:28.176178 17927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:28.177369 17927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:28.179031 17927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:28.191323 17935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:28.191349 17932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:28.191511 17933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:28.192056 17927 server_base.cc:1034] running on GCE node
I20260929 02:53:28.192603 17927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:28.193488 17927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:28.194698 17927 hybrid_clock.cc:648] HybridClock initialized: now 1790650408194630 us; error 45 us; skew 500 ppm
I20260929 02:53:28.194991 17927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:28.197165 17927 webserver.cc:460] Webserver started at http://127.8.57.2:45487/ using document root <none> and password file <none>
I20260929 02:53:28.197837 17927 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:28.197935 17927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:28.198223 17927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:28.200973 17927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "3f1f65965a5849faaa893fc26642c398"
format_stamp: "Formatted at 2026-09-29 02:53:28 on dist-test-slave-1dkt"
server_key: "f7243edb28207f0c31a9ee66e6728cd1"
server_key_iv: "51fb5cac51ba43a2755300f5d4105c3f"
server_key_version: "encryptionkey@0"
I20260929 02:53:28.201733 17927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "3f1f65965a5849faaa893fc26642c398"
format_stamp: "Formatted at 2026-09-29 02:53:28 on dist-test-slave-1dkt"
server_key: "f7243edb28207f0c31a9ee66e6728cd1"
server_key_iv: "51fb5cac51ba43a2755300f5d4105c3f"
server_key_version: "encryptionkey@0"
I20260929 02:53:28.206554 17927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 02:53:28.209779 17941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:28.211046 17927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:53:28.211191 17927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "3f1f65965a5849faaa893fc26642c398"
format_stamp: "Formatted at 2026-09-29 02:53:28 on dist-test-slave-1dkt"
server_key: "f7243edb28207f0c31a9ee66e6728cd1"
server_key_iv: "51fb5cac51ba43a2755300f5d4105c3f"
server_key_version: "encryptionkey@0"
I20260929 02:53:28.211375 17927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:28.226656 17927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:28.227429 17927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:28.227665 17927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:28.228547 17927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:28.229904 17927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:28.229993 17927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:28.230096 17927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:28.230162 17927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:28.258863 17927 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:44659
I20260929 02:53:28.258908 18053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:44659 every 8 connection(s)
I20260929 02:53:28.260139 17927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:53:28.263530 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 17927
I20260929 02:53:28.263672 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:53:28.264199 8420 external_mini_cluster.cc:1428] Setting key dd0e14f1020a55261b83c44ccc58a6fb
I20260929 02:53:28.266899 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37917
--builtin_ntp_servers=127.8.57.20:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 02:53:28.272615 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.262626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:34029 (local address 127.8.57.62:37917)
0929 02:53:28.262792 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:28.262797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:28.264317 (+ 1520us) server_negotiation.cc:407] Connection header received
0929 02:53:28.266352 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:28.266365 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:28.266440 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:28.266527 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:28.267638 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.268372 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.269905 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.270333 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.270442 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.271789 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:28.271794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:28.271797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:28.271815 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:28.271847 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:28.271859 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:28.271965 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:28.272169 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 02:53:28.272283 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:53:28.273718 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.263108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37917 (local address 127.8.57.2:34029)
0929 02:53:28.264142 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:28.264193 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:28.266060 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:28.266776 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:28.266784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:28.266877 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:28.267479 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:28.267496 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.268521 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.268525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:28.269768 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:28.269781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.270541 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.270548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:28.270778 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.271465 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:53:28.271469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:28.271511 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:53:28.271590 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:28.271961 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:28.271970 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:28.271979 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:53:28.272149 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 02:53:28.272570 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":295,"threads_started":1}
I20260929 02:53:28.275305 18054 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37917
I20260929 02:53:28.275611 18054 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:28.276577 18054 heartbeater.cc:511] Master 127.8.57.62:37917 requested a full tablet report, sending...
I20260929 02:53:28.278103 17737 ts_manager.cc:194] Registered new tserver with Master: 3f1f65965a5849faaa893fc26642c398 (127.8.57.2:44659)
I20260929 02:53:28.278929 17737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.2:34029
W20260929 02:53:28.582365 18057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:28.582720 18057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:28.582850 18057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:28.590339 18057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:28.590446 18057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:28.590586 18057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:53:28.599612 18057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:37917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:28.600891 18057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:28.602596 18057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:28.615309 18062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:28.615330 18063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:28.615286 18065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:28.616217 18057 server_base.cc:1034] running on GCE node
I20260929 02:53:28.616803 18057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:28.617769 18057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:28.618988 18057 hybrid_clock.cc:648] HybridClock initialized: now 1790650408618927 us; error 56 us; skew 500 ppm
I20260929 02:53:28.619364 18057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:28.621642 18057 webserver.cc:460] Webserver started at http://127.8.57.3:41669/ using document root <none> and password file <none>
I20260929 02:53:28.622331 18057 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:28.622431 18057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:28.622743 18057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:28.625633 18057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "0a90cb3eebf24ce4b5687eb86f5b6f7f"
format_stamp: "Formatted at 2026-09-29 02:53:28 on dist-test-slave-1dkt"
server_key: "c52d32ad30230a22d93831540f35bdfc"
server_key_iv: "bb1f94ebc7203fbef33a49c365eb5784"
server_key_version: "encryptionkey@0"
I20260929 02:53:28.626415 18057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "0a90cb3eebf24ce4b5687eb86f5b6f7f"
format_stamp: "Formatted at 2026-09-29 02:53:28 on dist-test-slave-1dkt"
server_key: "c52d32ad30230a22d93831540f35bdfc"
server_key_iv: "bb1f94ebc7203fbef33a49c365eb5784"
server_key_version: "encryptionkey@0"
I20260929 02:53:28.631542 18057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:53:28.635036 18071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:28.636449 18057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:28.636605 18057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "0a90cb3eebf24ce4b5687eb86f5b6f7f"
format_stamp: "Formatted at 2026-09-29 02:53:28 on dist-test-slave-1dkt"
server_key: "c52d32ad30230a22d93831540f35bdfc"
server_key_iv: "bb1f94ebc7203fbef33a49c365eb5784"
server_key_version: "encryptionkey@0"
I20260929 02:53:28.636806 18057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:28.655776 18057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:28.656667 18057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:28.656948 18057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:28.657867 18057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:28.659298 18057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:28.659397 18057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:28.659507 18057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:28.659574 18057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:28.688282 18057 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:42309
I20260929 02:53:28.688323 18183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:42309 every 8 connection(s)
I20260929 02:53:28.689481 18057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:53:28.698192 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18057
I20260929 02:53:28.698323 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:53:28.698735 8420 external_mini_cluster.cc:1428] Setting key ef0718871a092008f3121b7e251f97d6
I20260929 02:53:28.701153 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.691860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:32995 (local address 127.8.57.62:37917)
0929 02:53:28.692062 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:28.692067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:28.693388 (+ 1321us) server_negotiation.cc:407] Connection header received
0929 02:53:28.695069 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:28.695074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:28.695146 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:28.695236 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:28.696287 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.697032 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.698539 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.698984 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.699092 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.700378 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:28.700383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:28.700391 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:28.700408 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:28.700439 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:28.700450 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:28.700521 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:28.700740 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 02:53:28.700830 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:53:28.702112 18186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.692382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37917 (local address 127.8.57.3:32995)
0929 02:53:28.693244 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:28.693278 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:28.694796 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:28.695428 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:28.695436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:28.695525 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:28.696129 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:28.696148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.697138 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.697141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:28.698405 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:28.698417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.699212 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.699220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:28.699439 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.700076 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:53:28.700078 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:28.700116 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:53:28.700205 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:28.700530 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:28.700541 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:28.700548 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:53:28.700693 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 02:53:28.701081 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":244,"threads_started":1}
I20260929 02:53:28.703328 18184 heartbeater.cc:347] Connected to a master server at 127.8.57.62:37917
I20260929 02:53:28.703598 18184 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:28.704357 18184 heartbeater.cc:511] Master 127.8.57.62:37917 requested a full tablet report, sending...
I20260929 02:53:28.705626 17737 ts_manager.cc:194] Registered new tserver with Master: 0a90cb3eebf24ce4b5687eb86f5b6f7f (127.8.57.3:42309)
I20260929 02:53:28.706414 17737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.3:32995
I20260929 02:53:28.713423 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:53:28.719139 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:28.717666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.8.57.62:37917)
0929 02:53:28.717840 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:28.717847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:28.718256 (+ 409us) server_negotiation.cc:407] Connection header received
0929 02:53:28.718404 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:28.718408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:28.718478 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:28.718553 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:28.718961 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56764: BlockingRecv error: recv got EOF from 127.0.0.1:56764 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:53:28.719731 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17796
I20260929 02:53:28.726899 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.720537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.8.57.1:40819)
0929 02:53:28.721005 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:28.721012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:28.721034 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:53:28.721084 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:28.721090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:28.721303 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:28.721434 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:28.721817 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.722773 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.723956 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.724489 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.724597 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.725549 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:28.725556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:28.725569 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:28.726147 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:28.726200 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:28.726219 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:28.726315 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:28.726452 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 02:53:28.726592 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":199,"threads_started":1}
I20260929 02:53:28.728073 17918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:28.806902 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17796
I20260929 02:53:28.819644 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17927
I20260929 02:53:28.826855 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.820493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.8.57.2:44659)
0929 02:53:28.821004 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:28.821011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:28.821038 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:53:28.821102 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:28.821108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:28.821314 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:28.821444 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:28.821729 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.822759 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.823989 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.824564 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.824677 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.825569 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:28.825576 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:28.825590 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:28.826186 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:28.826237 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:28.826257 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:28.826341 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:28.826430 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 02:53:28.826549 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":263,"threads_started":1}
I20260929 02:53:28.828085 18049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:28.908749 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17927
I20260929 02:53:28.921326 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18057
I20260929 02:53:28.928973 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:28.922277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.8.57.3:42309)
0929 02:53:28.922866 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:28.922873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:28.922901 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:53:28.922961 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:28.922967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:28.923176 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:28.923300 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:28.923664 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.924761 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.925968 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:28.926563 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:28.926663 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:28.927598 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:28.927607 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:28.927620 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:28.928256 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:28.928305 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:28.928326 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:28.928411 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:28.928542 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 02:53:28.928651 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":285,"threads_started":1}
I20260929 02:53:28.930258 18179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:29.012121 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18057
I20260929 02:53:29.023968 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 17706
I20260929 02:53:29.025113 17767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:29.125676 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 17706
2026-09-29T02:53:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2283 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T02:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:29Z Disabled control of system clock
I20260929 02:53:29.194242 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:46439
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:44323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:46439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:53:29.508774 18203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:29.509060 18203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:29.509184 18203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:29.516765 18203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:29.516870 18203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:29.516927 18203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:29.517004 18203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:29.517062 18203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:29.526222 18203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:46439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:46439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:29.527467 18203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:29.529239 18203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:29.540266 18208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:29.540357 18211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:29.540292 18209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:29.540871 18203 server_base.cc:1034] running on GCE node
I20260929 02:53:29.541627 18203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:29.542965 18203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:29.544149 18203 hybrid_clock.cc:648] HybridClock initialized: now 1790650409544107 us; error 36 us; skew 500 ppm
I20260929 02:53:29.544515 18203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:29.546573 18203 webserver.cc:460] Webserver started at http://127.8.57.62:34551/ using document root <none> and password file <none>
I20260929 02:53:29.547225 18203 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:29.547314 18203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:29.547602 18203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:29.550449 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "45f1377304b64f608e5d8efbe14c86cc"
format_stamp: "Formatted at 2026-09-29 02:53:29 on dist-test-slave-1dkt"
server_key: "0d8787648fc55dfdabe6b29f6545d8f8"
server_key_iv: "58b11be60f1edce59fdd67ab93765380"
server_key_version: "encryptionkey@0"
I20260929 02:53:29.551192 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "45f1377304b64f608e5d8efbe14c86cc"
format_stamp: "Formatted at 2026-09-29 02:53:29 on dist-test-slave-1dkt"
server_key: "0d8787648fc55dfdabe6b29f6545d8f8"
server_key_iv: "58b11be60f1edce59fdd67ab93765380"
server_key_version: "encryptionkey@0"
I20260929 02:53:29.556118 18203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:53:29.559360 18217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:29.560709 18203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:29.560860 18203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "45f1377304b64f608e5d8efbe14c86cc"
format_stamp: "Formatted at 2026-09-29 02:53:29 on dist-test-slave-1dkt"
server_key: "0d8787648fc55dfdabe6b29f6545d8f8"
server_key_iv: "58b11be60f1edce59fdd67ab93765380"
server_key_version: "encryptionkey@0"
I20260929 02:53:29.561044 18203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:29.566406 18203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:29.567116 18203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:29.567391 18203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:29.583760 18203 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:46439
I20260929 02:53:29.583791 18268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:46439 every 8 connection(s)
I20260929 02:53:29.585057 18203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:53:29.588629 18269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:29.592566 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18203
I20260929 02:53:29.592777 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:53:29.593346 8420 external_mini_cluster.cc:1428] Setting key 27adad4ea5ef77d781cc98b54f6ff2d2
I20260929 02:53:29.599464 18269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc: Bootstrap starting.
I20260929 02:53:29.601292 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:29.595257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33524 (local address 127.8.57.62:46439)
0929 02:53:29.596250 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:29.596264 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:29.596307 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:53:29.597954 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:29.597975 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:29.598375 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:29.598685 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:29.599763 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:29.599770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:29.599782 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:29.599796 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:29.599862 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:29.599906 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:29.600077 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:29.600170 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 02:53:29.600430 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":297,"threads_started":1}
I20260929 02:53:29.602644 18269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:29.603753 18269 log.cc:824] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:29.606523 18269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc: No bootstrap required, opened a new log
I20260929 02:53:29.613590 18269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f1377304b64f608e5d8efbe14c86cc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46439 } }
I20260929 02:53:29.614002 18269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:29.614118 18269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45f1377304b64f608e5d8efbe14c86cc, State: Initialized, Role: FOLLOWER
I20260929 02:53:29.614691 18269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f1377304b64f608e5d8efbe14c86cc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46439 } }
I20260929 02:53:29.614883 18269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:53:29.614974 18269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:53:29.615089 18269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:29.617199 18269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f1377304b64f608e5d8efbe14c86cc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46439 } }
I20260929 02:53:29.617627 18269 leader_election.cc:302] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45f1377304b64f608e5d8efbe14c86cc; no voters:
I20260929 02:53:29.618113 18269 leader_election.cc:288] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:53:29.618232 18274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:29.618834 18274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [term 1 LEADER]: Becoming Leader. State: Replica: 45f1377304b64f608e5d8efbe14c86cc, State: Running, Role: LEADER
I20260929 02:53:29.619365 18274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f1377304b64f608e5d8efbe14c86cc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46439 } }
I20260929 02:53:29.620143 18269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:53:29.621970 18275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45f1377304b64f608e5d8efbe14c86cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f1377304b64f608e5d8efbe14c86cc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46439 } } }
I20260929 02:53:29.622365 18275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:29.621994 18276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45f1377304b64f608e5d8efbe14c86cc. Latest consensus state: current_term: 1 leader_uuid: "45f1377304b64f608e5d8efbe14c86cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f1377304b64f608e5d8efbe14c86cc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 46439 } } }
I20260929 02:53:29.624339 18276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:29.626470 18281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:53:29.631459 18281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:53:29.640321 18281 catalog_manager.cc:1347] Generated new cluster ID: b00b119f861d4580b0f9255ab606deab
I20260929 02:53:29.640431 18281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:53:29.658833 18281 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:53:29.659302 18281 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:53:29.667478 18281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45f1377304b64f608e5d8efbe14c86cc: Generated new TSK 0
I20260929 02:53:29.668565 18281 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:53:29.685842 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46439
--builtin_ntp_servers=127.8.57.20:44323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:53:30.005298 18293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:30.005591 18293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:30.005726 18293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:30.013600 18293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:30.013727 18293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:30.013896 18293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:53:30.023344 18293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:30.024669 18293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:30.026386 18293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:30.039304 18299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:30.039292 18298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:30.039566 18301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:30.040066 18293 server_base.cc:1034] running on GCE node
I20260929 02:53:30.040709 18293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:30.041664 18293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:30.042865 18293 hybrid_clock.cc:648] HybridClock initialized: now 1790650410042808 us; error 49 us; skew 500 ppm
I20260929 02:53:30.043169 18293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:30.045506 18293 webserver.cc:460] Webserver started at http://127.8.57.1:43331/ using document root <none> and password file <none>
I20260929 02:53:30.046205 18293 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:30.046310 18293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:30.046620 18293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:30.049547 18293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "4378f7e4d16842a2a37360abc900fec3"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "3e51768740428032063be463f502bdb3"
server_key_iv: "23b35476ea8a72f2300516c21a40a5ec"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.050341 18293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "4378f7e4d16842a2a37360abc900fec3"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "3e51768740428032063be463f502bdb3"
server_key_iv: "23b35476ea8a72f2300516c21a40a5ec"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.055495 18293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:53:30.059301 18307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.060778 18293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:53:30.060935 18293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "4378f7e4d16842a2a37360abc900fec3"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "3e51768740428032063be463f502bdb3"
server_key_iv: "23b35476ea8a72f2300516c21a40a5ec"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.061142 18293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:30.066830 18293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:30.067608 18293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:30.067847 18293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:30.068850 18293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:30.070163 18293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:30.070252 18293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.070333 18293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:30.070385 18293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.099462 18293 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:34523
I20260929 02:53:30.099514 18419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:34523 every 8 connection(s)
I20260929 02:53:30.100878 18293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:53:30.108678 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.103339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:38811 (local address 127.8.57.62:46439)
0929 02:53:30.103747 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:30.103752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:30.104951 (+ 1199us) server_negotiation.cc:407] Connection header received
0929 02:53:30.106583 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:30.106589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:30.106656 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:30.106755 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:30.107981 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:30.107987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:30.107992 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:30.108011 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:30.108054 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:30.108066 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:30.108167 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:30.108405 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 02:53:30.108459 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":200,"threads_started":1}
I20260929 02:53:30.108696 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18293
I20260929 02:53:30.108901 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:53:30.109658 8420 external_mini_cluster.cc:1428] Setting key 147b5cad6a68aa182c11ce49df289799
I20260929 02:53:30.109886 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.103941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46439 (local address 127.8.57.1:38811)
0929 02:53:30.104799 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:30.104839 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:30.106331 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:30.106989 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:30.106998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:30.107092 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:30.107689 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:53:30.107691 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:30.107727 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:53:30.107811 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:30.108150 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:30.108161 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:30.108168 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:53:30.108464 (+ 296us) client_negotiation.cc:240] Negotiation successful
0929 02:53:30.108725 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":242,"threads_started":1}
I20260929 02:53:30.111439 18420 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46439
I20260929 02:53:30.111752 18420 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:30.112326 18420 heartbeater.cc:511] Master 127.8.57.62:46439 requested a full tablet report, sending...
I20260929 02:53:30.112648 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46439
--builtin_ntp_servers=127.8.57.20:44323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 02:53:30.114275 18233 ts_manager.cc:194] Registered new tserver with Master: 4378f7e4d16842a2a37360abc900fec3 (127.8.57.1:34523)
W20260929 02:53:30.427142 18424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:30.427431 18424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:30.427541 18424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:30.434794 18424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:30.434893 18424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:30.435011 18424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:53:30.444120 18424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:30.445320 18424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:30.447098 18424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:30.459477 18429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:30.459826 18424 server_base.cc:1034] running on GCE node
W20260929 02:53:30.459524 18430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:30.459568 18432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:30.460647 18424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:30.461565 18424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:30.462756 18424 hybrid_clock.cc:648] HybridClock initialized: now 1790650410462706 us; error 42 us; skew 500 ppm
I20260929 02:53:30.463052 18424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:30.465346 18424 webserver.cc:460] Webserver started at http://127.8.57.2:39691/ using document root <none> and password file <none>
I20260929 02:53:30.466038 18424 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:30.466150 18424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:30.466466 18424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:30.470865 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "600895b3120c4d95a2a6b783384801b3"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "6b7415ef8383e23d1d2a43c9dbbddc14"
server_key_iv: "6237490a592955ade38f460e0f221673"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.471622 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "600895b3120c4d95a2a6b783384801b3"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "6b7415ef8383e23d1d2a43c9dbbddc14"
server_key_iv: "6237490a592955ade38f460e0f221673"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.476625 18424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:53:30.480001 18438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.481307 18424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:53:30.481459 18424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "600895b3120c4d95a2a6b783384801b3"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "6b7415ef8383e23d1d2a43c9dbbddc14"
server_key_iv: "6237490a592955ade38f460e0f221673"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.481652 18424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:30.487071 18424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:30.487849 18424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:30.488124 18424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:30.488986 18424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:30.490337 18424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:30.490428 18424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.490532 18424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:30.490599 18424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.518705 18424 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:37173
I20260929 02:53:30.518754 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:37173 every 8 connection(s)
I20260929 02:53:30.520010 18424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:53:30.522171 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18424
I20260929 02:53:30.522337 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:53:30.522778 8420 external_mini_cluster.cc:1428] Setting key 415e3fc5a9a9c817370069e3f197f63e
I20260929 02:53:30.526043 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46439
--builtin_ntp_servers=127.8.57.20:44323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 02:53:30.529091 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.522669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:49377 (local address 127.8.57.62:46439)
0929 02:53:30.522870 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:30.522877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:30.524670 (+ 1793us) server_negotiation.cc:407] Connection header received
0929 02:53:30.526919 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:30.526923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:30.526974 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:30.527055 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:30.528489 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:30.528493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:30.528496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:30.528511 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:30.528543 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:30.528554 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:30.528645 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:30.528818 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 02:53:30.528866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:53:30.530283 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.523251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46439 (local address 127.8.57.2:49377)
0929 02:53:30.524437 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:30.524483 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:30.526630 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:30.527328 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:30.527340 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:30.527474 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:30.528162 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:53:30.528165 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:30.528216 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:53:30.528333 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:30.528633 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:30.528640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:30.528646 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:53:30.528889 (+ 243us) client_negotiation.cc:240] Negotiation successful
0929 02:53:30.529153 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":320,"threads_started":1}
I20260929 02:53:30.531623 18551 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46439
I20260929 02:53:30.531942 18551 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:30.532603 18551 heartbeater.cc:511] Master 127.8.57.62:46439 requested a full tablet report, sending...
I20260929 02:53:30.534065 18233 ts_manager.cc:194] Registered new tserver with Master: 600895b3120c4d95a2a6b783384801b3 (127.8.57.2:37173)
W20260929 02:53:30.843459 18554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:30.843736 18554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:30.843851 18554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:30.851156 18554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:30.851258 18554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:30.851408 18554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:53:30.860886 18554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:46439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:30.862125 18554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:30.863821 18554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:30.876678 18559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:30.876645 18560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:30.876721 18562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:30.877516 18554 server_base.cc:1034] running on GCE node
I20260929 02:53:30.878089 18554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:30.879012 18554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:30.880208 18554 hybrid_clock.cc:648] HybridClock initialized: now 1790650410880128 us; error 66 us; skew 500 ppm
I20260929 02:53:30.880512 18554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:53:30.882699 18554 webserver.cc:460] Webserver started at http://127.8.57.3:42871/ using document root <none> and password file <none>
I20260929 02:53:30.883404 18554 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:30.883496 18554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:30.883826 18554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:30.887099 18554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "7859bf16d7e04a28bf1678177c92f7e5"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "20bfdacf0d8fd9bf928b598902df598d"
server_key_iv: "1c0a203fec8d6bb52d89b1ee018b45ca"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.887852 18554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "7859bf16d7e04a28bf1678177c92f7e5"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "20bfdacf0d8fd9bf928b598902df598d"
server_key_iv: "1c0a203fec8d6bb52d89b1ee018b45ca"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.892959 18554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:53:30.896375 18568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.897680 18554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 02:53:30.897827 18554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "7859bf16d7e04a28bf1678177c92f7e5"
format_stamp: "Formatted at 2026-09-29 02:53:30 on dist-test-slave-1dkt"
server_key: "20bfdacf0d8fd9bf928b598902df598d"
server_key_iv: "1c0a203fec8d6bb52d89b1ee018b45ca"
server_key_version: "encryptionkey@0"
I20260929 02:53:30.898028 18554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:30.903481 18554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:30.904294 18554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:30.904520 18554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:30.905361 18554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:30.906723 18554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:30.906807 18554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.906908 18554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:30.906975 18554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:30.935865 18554 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:38991
I20260929 02:53:30.935949 18680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:38991 every 8 connection(s)
I20260929 02:53:30.937181 18554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:53:30.938280 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18554
I20260929 02:53:30.938458 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:53:30.939003 8420 external_mini_cluster.cc:1428] Setting key 0a95f0e527a5f395b8a173a328f573a7
I20260929 02:53:30.945401 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.939814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:43175 (local address 127.8.57.62:46439)
0929 02:53:30.940020 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:30.940025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:30.941591 (+ 1566us) server_negotiation.cc:407] Connection header received
0929 02:53:30.943364 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:30.943369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:30.943434 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:30.943512 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:30.944737 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:30.944751 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:30.944755 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:30.944771 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:30.944801 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:30.944813 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:30.944914 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:30.945084 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 02:53:30.945146 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:53:30.946357 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.940462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:46439 (local address 127.8.57.3:43175)
0929 02:53:30.941452 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:30.941492 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:30.943113 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:30.943751 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:30.943760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:30.943854 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:30.944467 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:53:30.944469 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:30.944505 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:53:30.944587 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:30.944894 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:30.944904 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:30.944912 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:53:30.945156 (+ 244us) client_negotiation.cc:240] Negotiation successful
0929 02:53:30.945419 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":278,"threads_started":1}
I20260929 02:53:30.947561 18681 heartbeater.cc:347] Connected to a master server at 127.8.57.62:46439
I20260929 02:53:30.947814 18681 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:30.948371 18681 heartbeater.cc:511] Master 127.8.57.62:46439 requested a full tablet report, sending...
I20260929 02:53:30.949702 18233 ts_manager.cc:194] Registered new tserver with Master: 7859bf16d7e04a28bf1678177c92f7e5 (127.8.57.3:38991)
I20260929 02:53:30.954154 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:53:30.959671 18422 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:53:30.958432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.8.57.62:46439)
0929 02:53:30.958571 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:30.958575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:30.959031 (+ 456us) server_negotiation.cc:407] Connection header received
0929 02:53:30.959148 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:30.959152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:30.959202 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:30.959275 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:30.959496 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33526: BlockingRecv error: recv got EOF from 127.0.0.1:33526 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:53:30.960110 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18293
I20260929 02:53:30.963330 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:30.961173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.8.57.1:34523)
0929 02:53:30.961647 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:30.961654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:30.961679 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:53:30.961733 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:30.961746 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:30.961938 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:30.962052 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:30.962805 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:30.962812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:30.962822 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:30.962840 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:30.962879 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:30.962892 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:30.962989 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:30.963038 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 02:53:30.963110 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":232,"threads_started":1}
I20260929 02:53:30.964550 18415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:31.045047 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18293
I20260929 02:53:31.057971 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18424
I20260929 02:53:31.061362 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:31.058855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.8.57.2:37173)
0929 02:53:31.059355 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:31.059362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:31.059391 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:53:31.059451 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:31.059458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:31.059665 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:31.059799 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:31.060696 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:31.060702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:31.060721 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:31.060737 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:31.060777 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:31.060791 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:31.060896 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:31.061030 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 02:53:31.061113 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":243,"threads_started":1}
I20260929 02:53:31.062558 18546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:31.143797 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18424
I20260929 02:53:31.155846 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18554
I20260929 02:53:31.159709 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:31.156814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.8.57.3:38991)
0929 02:53:31.157415 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:31.157421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:31.157456 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:53:31.157527 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:31.157536 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:31.157775 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:31.157917 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:31.158991 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:31.158996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:31.159008 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:53:31.159023 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:31.159059 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:53:31.159072 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:31.159219 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:31.159295 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 02:53:31.159392 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":283,"threads_started":1}
I20260929 02:53:31.160874 18676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:31.240780 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18554
I20260929 02:53:31.252733 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18203
I20260929 02:53:31.253857 18264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:31.353633 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18203
2026-09-29T02:53:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2222 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:53:31 dist-test-slave-1dkt krb5kdc[18699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:53:31 dist-test-slave-1dkt krb5kdc[18699](info): set up 2 sockets
Sep 29 02:53:31 dist-test-slave-1dkt krb5kdc[18699](info): commencing operation
krb5kdc: starting...
W20260929 02:53:32.360730 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:53:32 dist-test-slave-1dkt krb5kdc[18699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:53:32.581691 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:44909
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:44909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:32.896775 18712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:32.897089 18712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:32.897228 18712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:32.904534 18712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:53:32.904662 18712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:32.904742 18712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:32.904819 18712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:53:32.904879 18712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:53:32.914848 18712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:44909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:44909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:32.916234 18712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:32.917513 18712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:32.928323 18717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:32.928390 18718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:32.928409 18720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:32.929245 18712 server_base.cc:1034] running on GCE node
I20260929 02:53:32.929924 18712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:32.931329 18712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:32.932533 18712 hybrid_clock.cc:648] HybridClock initialized: now 1790650412932470 us; error 43 us; skew 500 ppm
I20260929 02:53:32.932883 18712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:32 dist-test-slave-1dkt krb5kdc[18699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:32.935794 18712 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:53:32.937206 18712 webserver.cc:460] Webserver started at http://127.8.57.62:46155/ using document root <none> and password file <none>
I20260929 02:53:32.937881 18712 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:32.937983 18712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:32.938277 18712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:32.941287 18712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "a61b0d49f67d4e09b64044518e8e8582"
format_stamp: "Formatted at 2026-09-29 02:53:32 on dist-test-slave-1dkt"
server_key: "dbfc544e4a96b4c8c6f48ff8430d982d"
server_key_iv: "977c39409b3e7e6ff379566a7a5af39a"
server_key_version: "encryptionkey@0"
I20260929 02:53:32.942003 18712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "a61b0d49f67d4e09b64044518e8e8582"
format_stamp: "Formatted at 2026-09-29 02:53:32 on dist-test-slave-1dkt"
server_key: "dbfc544e4a96b4c8c6f48ff8430d982d"
server_key_iv: "977c39409b3e7e6ff379566a7a5af39a"
server_key_version: "encryptionkey@0"
I20260929 02:53:32.946991 18712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:53:32.950369 18727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:32.951675 18712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:53:32.951835 18712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "a61b0d49f67d4e09b64044518e8e8582"
format_stamp: "Formatted at 2026-09-29 02:53:32 on dist-test-slave-1dkt"
server_key: "dbfc544e4a96b4c8c6f48ff8430d982d"
server_key_iv: "977c39409b3e7e6ff379566a7a5af39a"
server_key_version: "encryptionkey@0"
I20260929 02:53:32.952052 18712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:32.979048 18712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:32.983383 18712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:32.983652 18712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:33.000417 18712 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:44909
I20260929 02:53:33.000439 18778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:44909 every 8 connection(s)
I20260929 02:53:33.001722 18712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:53:33.002208 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18712
I20260929 02:53:33.002417 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:53:33.003125 8420 external_mini_cluster.cc:1428] Setting key f1d67e6460bc9ee2ecdea5d26927b207
I20260929 02:53:33.005959 18779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:53:33 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:33.016482 18779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582: Bootstrap starting.
I20260929 02:53:33.019531 18779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:33.020612 18779 log.cc:824] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:33.023437 18779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582: No bootstrap required, opened a new log
I20260929 02:53:33.024678 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:33.005044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40286 (local address 127.8.57.62:44909)
0929 02:53:33.005824 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:33.005837 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:33.005878 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:53:33.007690 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:33.007714 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:33.008184 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:33.008579 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:33.010045 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:33.011408 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:33.012744 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:33.013418 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:33.013611 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:33.017555 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:33.017563 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:33.017578 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:33.017617 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:33.020627 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:33.021408 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:33.021415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:33.021421 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:33.021581 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:33.022000 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:33.022004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:33.022006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:33.022531 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:33.022844 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:33.023236 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 02:53:33.023660 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":343,"threads_started":1}
I20260929 02:53:33.031221 18779 raft_consensus.cc:348] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61b0d49f67d4e09b64044518e8e8582" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 44909 } }
I20260929 02:53:33.031570 18779 raft_consensus.cc:374] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:33.031670 18779 raft_consensus.cc:729] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a61b0d49f67d4e09b64044518e8e8582, State: Initialized, Role: FOLLOWER
I20260929 02:53:33.032279 18779 consensus_queue.cc:260] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61b0d49f67d4e09b64044518e8e8582" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 44909 } }
I20260929 02:53:33.032545 18779 raft_consensus.cc:388] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:53:33.032688 18779 raft_consensus.cc:482] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:53:33.032856 18779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:33.034901 18779 raft_consensus.cc:504] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61b0d49f67d4e09b64044518e8e8582" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 44909 } }
I20260929 02:53:33.035367 18779 leader_election.cc:302] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a61b0d49f67d4e09b64044518e8e8582; no voters:
I20260929 02:53:33.035988 18779 leader_election.cc:288] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:53:33.036087 18784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:33.036717 18784 raft_consensus.cc:686] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [term 1 LEADER]: Becoming Leader. State: Replica: a61b0d49f67d4e09b64044518e8e8582, State: Running, Role: LEADER
I20260929 02:53:33.037380 18784 consensus_queue.cc:237] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61b0d49f67d4e09b64044518e8e8582" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 44909 } }
I20260929 02:53:33.038000 18779 sys_catalog.cc:564] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:53:33.040091 18786 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a61b0d49f67d4e09b64044518e8e8582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61b0d49f67d4e09b64044518e8e8582" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 44909 } } }
I20260929 02:53:33.040459 18786 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:33.040275 18785 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a61b0d49f67d4e09b64044518e8e8582. Latest consensus state: current_term: 1 leader_uuid: "a61b0d49f67d4e09b64044518e8e8582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61b0d49f67d4e09b64044518e8e8582" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 44909 } } }
I20260929 02:53:33.040860 18785 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582 [sys.catalog]: This master's current role is: LEADER
I20260929 02:53:33.041893 18793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:53:33.047132 18793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:53:33.055404 18793 catalog_manager.cc:1347] Generated new cluster ID: 00f945b043244b16bb56e690cf79e90b
I20260929 02:53:33.055523 18793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:53:33.089250 18793 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:53:33.091131 18793 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:53:33.097424 18793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582: Generated new TSK 0
I20260929 02:53:33.098429 18793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:53:33.159248 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:44909
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:33.475932 18805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:33.476217 18805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:33.476310 18805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:33.483438 18805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:33.483549 18805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:33.483660 18805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:53:33.493424 18805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:44909
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:33.494629 18805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:33.495894 18805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:33.508831 18813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:33.508890 18810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:33.508886 18811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:33.509392 18805 server_base.cc:1034] running on GCE node
I20260929 02:53:33.510018 18805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:33.510983 18805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:33.512187 18805 hybrid_clock.cc:648] HybridClock initialized: now 1790650413512120 us; error 50 us; skew 500 ppm
I20260929 02:53:33.512509 18805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:33 dist-test-slave-1dkt krb5kdc[18699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:33.515455 18805 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:53:33.516906 18805 webserver.cc:460] Webserver started at http://127.8.57.1:34913/ using document root <none> and password file <none>
I20260929 02:53:33.517566 18805 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:33.517691 18805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:33.517970 18805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:33.520916 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "e08d42db15be49339719d32f8d109504"
format_stamp: "Formatted at 2026-09-29 02:53:33 on dist-test-slave-1dkt"
server_key: "53dad77ca21ddf0ca3352b56ad506dca"
server_key_iv: "03e0c9078e860bff558eb14dc014fd0f"
server_key_version: "encryptionkey@0"
I20260929 02:53:33.521664 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "e08d42db15be49339719d32f8d109504"
format_stamp: "Formatted at 2026-09-29 02:53:33 on dist-test-slave-1dkt"
server_key: "53dad77ca21ddf0ca3352b56ad506dca"
server_key_iv: "03e0c9078e860bff558eb14dc014fd0f"
server_key_version: "encryptionkey@0"
I20260929 02:53:33.526614 18805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 02:53:33.529989 18820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:33.531271 18805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:53:33.531417 18805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "e08d42db15be49339719d32f8d109504"
format_stamp: "Formatted at 2026-09-29 02:53:33 on dist-test-slave-1dkt"
server_key: "53dad77ca21ddf0ca3352b56ad506dca"
server_key_iv: "03e0c9078e860bff558eb14dc014fd0f"
server_key_version: "encryptionkey@0"
I20260929 02:53:33.531597 18805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:33.550033 18805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:33.554107 18805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:33.554450 18805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:33.555393 18805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:33.556804 18805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:33.556876 18805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:33.556967 18805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:33.557008 18805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:33.586588 18805 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:37923
I20260929 02:53:33.586645 18932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:37923 every 8 connection(s)
I20260929 02:53:33.587829 18805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:53:33.589777 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18805
I20260929 02:53:33.589985 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:53:33.590658 8420 external_mini_cluster.cc:1428] Setting key 79f0fd568837f526891f017c877a47e0
Sep 29 02:53:33 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:33.608111 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:33.590484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:53679 (local address 127.8.57.62:44909)
0929 02:53:33.590946 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:33.590953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:33.591911 (+ 958us) server_negotiation.cc:407] Connection header received
0929 02:53:33.593739 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:33.593745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:33.593828 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:33.593921 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:33.595369 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:33.596108 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:33.597877 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:33.598299 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:33.598421 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:33.602043 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:33.602049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:33.602053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:33.602089 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:33.604937 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:33.605655 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:33.605660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:33.605663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:33.605796 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:33.606254 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:33.606258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:33.606262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:33.606608 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:33.606766 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:33.607463 (+ 697us) server_negotiation.cc:299] Negotiation successful
0929 02:53:33.607707 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":207,"threads_started":1}
I20260929 02:53:33.609004 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:33.591007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:44909 (local address 127.8.57.1:53679)
0929 02:53:33.591747 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:33.591792 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:33.593488 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:33.594166 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:33.594175 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:33.594650 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:33.595204 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:33.595230 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:33.596253 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:33.596257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:33.597747 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:33.597761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:33.598512 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:33.598517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:33.598723 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:33.599375 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:33.599401 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:33.601817 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:33.605083 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:33.605089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:33.605103 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:33.605546 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:33.605926 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:33.605930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:33.605932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:33.606145 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:33.606757 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:33.606764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:33.607101 (+ 337us) client_negotiation.cc:769] Sending connection context
0929 02:53:33.607355 (+ 254us) client_negotiation.cc:240] Negotiation successful
0929 02:53:33.607822 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":273,"threads_started":1}
I20260929 02:53:33.610530 18933 heartbeater.cc:347] Connected to a master server at 127.8.57.62:44909
I20260929 02:53:33.610878 18933 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:53:33.611829 18933 heartbeater.cc:511] Master 127.8.57.62:44909 requested a full tablet report, sending...
I20260929 02:53:33.614081 18744 ts_manager.cc:194] Registered new tserver with Master: e08d42db15be49339719d32f8d109504 (127.8.57.1:37923)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:53:33.615934 18744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:53679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:53:33.641448 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:44909
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:33.956629 18939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:33.956936 18939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:33.957074 18939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:33.964432 18939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:33.964537 18939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:33.964680 18939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:53:33.974649 18939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:44909
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:33.975975 18939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:33.977290 18939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:33.989912 18944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:33.990015 18945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:33.989948 18947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:33.990880 18939 server_base.cc:1034] running on GCE node
I20260929 02:53:33.991463 18939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:33.992425 18939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:33.993634 18939 hybrid_clock.cc:648] HybridClock initialized: now 1790650413993585 us; error 34 us; skew 500 ppm
I20260929 02:53:33.993966 18939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:33 dist-test-slave-1dkt krb5kdc[18699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:33.996940 18939 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:53:33.998350 18939 webserver.cc:460] Webserver started at http://127.8.57.2:39301/ using document root <none> and password file <none>
I20260929 02:53:33.998979 18939 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:33.999078 18939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:33.999336 18939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:34.003075 18939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "344b76a2a3ac4148881553b1856c0051"
format_stamp: "Formatted at 2026-09-29 02:53:33 on dist-test-slave-1dkt"
server_key: "83d92af4cc933e7bcccb5543c9cd698f"
server_key_iv: "4fd9b240de7c01e7d4b72433ba96eefa"
server_key_version: "encryptionkey@0"
I20260929 02:53:34.004089 18939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "344b76a2a3ac4148881553b1856c0051"
format_stamp: "Formatted at 2026-09-29 02:53:33 on dist-test-slave-1dkt"
server_key: "83d92af4cc933e7bcccb5543c9cd698f"
server_key_iv: "4fd9b240de7c01e7d4b72433ba96eefa"
server_key_version: "encryptionkey@0"
I20260929 02:53:34.010574 18939 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 02:53:34.014971 18954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:34.016672 18939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:53:34.016831 18939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "344b76a2a3ac4148881553b1856c0051"
format_stamp: "Formatted at 2026-09-29 02:53:33 on dist-test-slave-1dkt"
server_key: "83d92af4cc933e7bcccb5543c9cd698f"
server_key_iv: "4fd9b240de7c01e7d4b72433ba96eefa"
server_key_version: "encryptionkey@0"
I20260929 02:53:34.017048 18939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:34.045512 18939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:34.049677 18939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:34.049921 18939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:34.050812 18939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:34.052175 18939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:34.052248 18939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:34.052343 18939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:34.052386 18939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:34.081880 18939 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:43631
I20260929 02:53:34.081965 19066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:43631 every 8 connection(s)
I20260929 02:53:34.083128 18939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:53:34.092665 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 18939
I20260929 02:53:34.092852 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:53:34.093395 8420 external_mini_cluster.cc:1428] Setting key a9f300dee6b91451e6e17f69e3e743a5
Sep 29 02:53:34 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:34.103233 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.085742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:40295 (local address 127.8.57.62:44909)
0929 02:53:34.085947 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.085954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.087097 (+ 1143us) server_negotiation.cc:407] Connection header received
0929 02:53:34.088936 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.088942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.089025 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.089115 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:34.090665 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.091409 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.093062 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.093708 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.093847 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.097481 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:34.097488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:34.097493 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:34.097533 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:34.100464 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.101154 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.101160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.101163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.101311 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.101687 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.101691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.101694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.101996 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:34.102127 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.102742 (+ 615us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.102968 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:53:34.104410 19069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.086218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:44909 (local address 127.8.57.2:40295)
0929 02:53:34.086935 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.086979 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.088637 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.089349 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.089361 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.089899 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:34.090473 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.090491 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.091540 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.091544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.092894 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.092910 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.093910 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.093915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.094177 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.094811 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:34.094834 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:34.097253 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:34.100588 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:34.100594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:34.100607 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:34.101033 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:34.101413 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:34.101416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:34.101418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:34.101588 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:34.102102 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:34.102107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:34.102406 (+ 299us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.102616 (+ 210us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.103024 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":274,"threads_started":1}
I20260929 02:53:34.105791 19067 heartbeater.cc:347] Connected to a master server at 127.8.57.62:44909
I20260929 02:53:34.106066 19067 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:34.106781 19067 heartbeater.cc:511] Master 127.8.57.62:44909 requested a full tablet report, sending...
I20260929 02:53:34.108145 18744 ts_manager.cc:194] Registered new tserver with Master: 344b76a2a3ac4148881553b1856c0051 (127.8.57.2:43631)
I20260929 02:53:34.108901 18744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:40295
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:53:34.141130 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:44909
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:53:34.451810 19072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:53:34.452123 19072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:53:34.452268 19072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:53:34.459357 19072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:53:34.459463 19072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:53:34.459613 19072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:53:34.468873 19072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:44909
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:53:34.470033 19072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:53:34.471267 19072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:53:34.483263 19080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:34.483274 19077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:53:34.483284 19078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:53:34.484242 19072 server_base.cc:1034] running on GCE node
I20260929 02:53:34.484787 19072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:53:34.485615 19072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:53:34.486799 19072 hybrid_clock.cc:648] HybridClock initialized: now 1790650414486748 us; error 40 us; skew 500 ppm
I20260929 02:53:34.487084 19072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:53:34 dist-test-slave-1dkt krb5kdc[18699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:53:34.490046 19072 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:53:34.491358 19072 webserver.cc:460] Webserver started at http://127.8.57.3:46315/ using document root <none> and password file <none>
I20260929 02:53:34.492036 19072 fs_manager.cc:359] Metadata directory not provided
I20260929 02:53:34.492149 19072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:53:34.492417 19072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:53:34.495234 19072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "5acd03e4f8c0416ea94cef36dcd678e1"
format_stamp: "Formatted at 2026-09-29 02:53:34 on dist-test-slave-1dkt"
server_key: "8f48da97aee56b0ff656348f2009a30d"
server_key_iv: "68cab009c2d5b23b018183a87ff3a45b"
server_key_version: "encryptionkey@0"
I20260929 02:53:34.495997 19072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "5acd03e4f8c0416ea94cef36dcd678e1"
format_stamp: "Formatted at 2026-09-29 02:53:34 on dist-test-slave-1dkt"
server_key: "8f48da97aee56b0ff656348f2009a30d"
server_key_iv: "68cab009c2d5b23b018183a87ff3a45b"
server_key_version: "encryptionkey@0"
I20260929 02:53:34.500859 19072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:53:34.504065 19087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:34.505328 19072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:53:34.505448 19072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "5acd03e4f8c0416ea94cef36dcd678e1"
format_stamp: "Formatted at 2026-09-29 02:53:34 on dist-test-slave-1dkt"
server_key: "8f48da97aee56b0ff656348f2009a30d"
server_key_iv: "68cab009c2d5b23b018183a87ff3a45b"
server_key_version: "encryptionkey@0"
I20260929 02:53:34.505602 19072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:53:34.516688 19072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:53:34.520265 19072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:53:34.520483 19072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:53:34.521320 19072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:53:34.522523 19072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:53:34.522593 19072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:34.522675 19072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:53:34.522715 19072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:53:34.550477 19072 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:33893
I20260929 02:53:34.550520 19199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:33893 every 8 connection(s)
I20260929 02:53:34.551625 19072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:53:34.560056 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 19072
I20260929 02:53:34.560218 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:53:34.560607 8420 external_mini_cluster.cc:1428] Setting key a562f0bd84cf4125dc7c1ea50a238927
Sep 29 02:53:34 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:53:34.569617 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.553983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:33315 (local address 127.8.57.62:44909)
0929 02:53:34.554169 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.554174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.555261 (+ 1087us) server_negotiation.cc:407] Connection header received
0929 02:53:34.557047 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.557051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.557116 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.557206 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:34.558594 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.559345 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.560922 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.561432 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.561549 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.564762 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:34.564766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:34.564770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:34.564801 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:34.567047 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.567713 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.567716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.567718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.567843 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.568235 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.568239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.568242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.568493 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:34.568599 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.569213 (+ 614us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.569381 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:53:34.570513 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.554465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:44909 (local address 127.8.57.3:33315)
0929 02:53:34.555131 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.555167 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.556818 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.557409 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.557417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.557906 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:53:34.558467 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.558483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.559472 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.559476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.560789 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.560800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.561636 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.561643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.561867 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.562458 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:53:34.562478 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:53:34.564585 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:53:34.567212 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:34.567219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:34.567231 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:34.567619 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:34.567961 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:53:34.567965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:53:34.567968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:53:34.568140 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:53:34.568588 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:53:34.568593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:53:34.568879 (+ 286us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.569100 (+ 221us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.569459 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":250,"threads_started":1}
I20260929 02:53:34.571841 19200 heartbeater.cc:347] Connected to a master server at 127.8.57.62:44909
I20260929 02:53:34.572121 19200 heartbeater.cc:464] Registering TS with master...
I20260929 02:53:34.572786 19200 heartbeater.cc:511] Master 127.8.57.62:44909 requested a full tablet report, sending...
I20260929 02:53:34.574074 18744 ts_manager.cc:194] Registered new tserver with Master: 5acd03e4f8c0416ea94cef36dcd678e1 (127.8.57.3:33893)
I20260929 02:53:34.574950 18744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:33315
I20260929 02:53:34.575452 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:53:34.591066 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.579656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.8.57.62:44909)
0929 02:53:34.579852 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.579859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.580241 (+ 382us) server_negotiation.cc:407] Connection header received
0929 02:53:34.580367 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.580370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.580427 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.580499 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:34.581766 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.582436 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.583548 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.584002 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.584091 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.586234 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:34.586240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:34.586244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:34.586279 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:34.588672 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.589268 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.589273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.589277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.589390 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.589743 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.589748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.589750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.590033 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:34.590243 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.590576 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.590794 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:53:34.594679 18744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:53:34.601157 18744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:53:34.619184 18933 heartbeater.cc:503] Master 127.8.57.62:44909 was elected leader, sending a full tablet report...
I20260929 02:53:34.622504 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.611818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33893 (local address 127.0.0.1:33322)
0929 02:53:34.612654 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.612692 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.612853 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.614433 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.614441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.614472 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:34.615005 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.615017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.617337 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.617342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.619730 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.619743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.621555 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.621562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.621761 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.621812 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.621917 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.622086 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":73,"thread_start_us":1459,"threads_started":1}
I20260929 02:53:34.623515 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.612374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.8.57.2:43631)
0929 02:53:34.613368 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.613376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.613541 (+ 165us) server_negotiation.cc:407] Connection header received
0929 02:53:34.613843 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.613851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.614087 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.614253 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:34.615284 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.617006 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.621036 (+ 4030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.622282 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.622353 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.622495 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.623046 (+ 551us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.623203 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":726,"threads_started":1}
I20260929 02:53:34.623557 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.612045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43631 (local address 127.0.0.1:46194)
0929 02:53:34.613420 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.613439 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.613639 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.614556 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.614561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.614593 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:34.615105 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.615116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.617192 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.617198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.620859 (+ 3661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.620872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.622683 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.622689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.622882 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.622909 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.622985 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.623180 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1241,"mutex_wait_us":830,"thread_start_us":383,"threads_started":1}
I20260929 02:53:34.623767 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.612061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33322 (local address 127.8.57.3:33893)
0929 02:53:34.613818 (+ 1757us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.613827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.613860 (+ 33us) server_negotiation.cc:407] Connection header received
0929 02:53:34.613957 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.613965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.614190 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.615207 (+ 1017us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:34.615472 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.617166 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.619948 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.621329 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.623010 (+ 1681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.623220 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.623304 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.623455 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1594,"thread_start_us":1447,"threads_started":1}
I20260929 02:53:34.626799 19216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.613620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37923 (local address 127.0.0.1:40610)
0929 02:53:34.618324 (+ 4704us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.618350 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.618466 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.620084 (+ 1618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.620089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.620113 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:34.620521 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.620535 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.622680 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.622685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.624640 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.624654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.626069 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.626076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.626275 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.626307 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.626391 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.626502 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4558,"thread_start_us":4394,"threads_started":1}
I20260929 02:53:34.626906 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.613989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.8.57.1:37923)
0929 02:53:34.619344 (+ 5355us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.619351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.619376 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:53:34.619456 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.619467 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.619685 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.619862 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:34.620741 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.622519 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.624809 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.625894 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.625989 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.626201 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.626470 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.626620 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5192,"thread_start_us":5047,"threads_started":1}
I20260929 02:53:34.627404 19002 tablet_service.cc:1467] Processing CreateTablet for tablet 7e967ff781ad40a9adb83c88f03c0b86 (DEFAULT_TABLE table=test-table [id=e328a1bb80fd438980a50844684afcf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:34.628782 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e967ff781ad40a9adb83c88f03c0b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:34.628500 19135 tablet_service.cc:1467] Processing CreateTablet for tablet 7e967ff781ad40a9adb83c88f03c0b86 (DEFAULT_TABLE table=test-table [id=e328a1bb80fd438980a50844684afcf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:34.629870 19135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e967ff781ad40a9adb83c88f03c0b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:34.630259 18868 tablet_service.cc:1467] Processing CreateTablet for tablet 7e967ff781ad40a9adb83c88f03c0b86 (DEFAULT_TABLE table=test-table [id=e328a1bb80fd438980a50844684afcf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:53:34.631474 18868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e967ff781ad40a9adb83c88f03c0b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:53:34.640872 19218 tablet_bootstrap.cc:492] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Bootstrap starting.
I20260929 02:53:34.643412 19219 tablet_bootstrap.cc:492] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Bootstrap starting.
I20260929 02:53:34.644249 19218 tablet_bootstrap.cc:654] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:34.644853 19220 tablet_bootstrap.cc:492] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Bootstrap starting.
I20260929 02:53:34.645525 19218 log.cc:824] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:34.646842 19219 tablet_bootstrap.cc:654] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:34.647920 19219 log.cc:824] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:34.648491 19220 tablet_bootstrap.cc:654] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Neither blocks nor log segments found. Creating new log.
I20260929 02:53:34.648622 19218 tablet_bootstrap.cc:492] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: No bootstrap required, opened a new log
I20260929 02:53:34.648872 19218 ts_tablet_manager.cc:1397] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 02:53:34.649590 19220 log.cc:824] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Log is configured to *not* fsync() on all Append() calls
I20260929 02:53:34.650245 19219 tablet_bootstrap.cc:492] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: No bootstrap required, opened a new log
I20260929 02:53:34.650488 19219 ts_tablet_manager.cc:1397] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:53:34.652305 19220 tablet_bootstrap.cc:492] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: No bootstrap required, opened a new log
I20260929 02:53:34.652546 19220 ts_tablet_manager.cc:1397] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:53:34.657618 19219 raft_consensus.cc:348] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.657958 19219 raft_consensus.cc:374] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:34.658077 19219 raft_consensus.cc:729] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acd03e4f8c0416ea94cef36dcd678e1, State: Initialized, Role: FOLLOWER
I20260929 02:53:34.658610 19219 consensus_queue.cc:260] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.658699 19218 raft_consensus.cc:348] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.659135 19218 raft_consensus.cc:374] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:34.659261 19218 raft_consensus.cc:729] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344b76a2a3ac4148881553b1856c0051, State: Initialized, Role: FOLLOWER
I20260929 02:53:34.659657 19200 heartbeater.cc:503] Master 127.8.57.62:44909 was elected leader, sending a full tablet report...
I20260929 02:53:34.659945 19218 consensus_queue.cc:260] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.660290 19219 ts_tablet_manager.cc:1428] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260929 02:53:34.661139 19067 heartbeater.cc:503] Master 127.8.57.62:44909 was elected leader, sending a full tablet report...
I20260929 02:53:34.661685 19218 ts_tablet_manager.cc:1428] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260929 02:53:34.662405 19220 raft_consensus.cc:348] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.662765 19220 raft_consensus.cc:374] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:53:34.662863 19220 raft_consensus.cc:729] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08d42db15be49339719d32f8d109504, State: Initialized, Role: FOLLOWER
I20260929 02:53:34.663391 19220 consensus_queue.cc:260] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.664949 19220 ts_tablet_manager.cc:1428] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260929 02:53:34.678561 19226 raft_consensus.cc:482] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:53:34.678773 19226 raft_consensus.cc:504] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.680336 19226 leader_election.cc:288] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5acd03e4f8c0416ea94cef36dcd678e1 (127.8.57.3:33893), 344b76a2a3ac4148881553b1856c0051 (127.8.57.2:43631)
I20260929 02:53:34.686455 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.680678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:34725 (local address 127.8.57.3:33893)
0929 02:53:34.680862 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.680867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.681599 (+ 732us) server_negotiation.cc:407] Connection header received
0929 02:53:34.681766 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.681769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.681824 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.681891 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:34.682443 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.683158 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.684901 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.685620 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.685727 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.685832 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.686100 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.686227 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:53:34.686549 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.680804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:39013 (local address 127.8.57.2:43631)
0929 02:53:34.680990 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.680996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.681674 (+ 678us) server_negotiation.cc:407] Connection header received
0929 02:53:34.681851 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.681854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.681907 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.681976 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:53:34.682563 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.683223 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.684956 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.685710 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.685826 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.685913 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.686196 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.686321 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:53:34.686646 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.680488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:33893 (local address 127.8.57.1:34725)
0929 02:53:34.681505 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.681520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.681630 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.681919 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.681923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.681945 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:34.682315 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.682325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.683279 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.683282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.684764 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.684776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.685804 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.685809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.685966 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.685990 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.686065 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.686195 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":38,"thread_start_us":606,"threads_started":1}
I20260929 02:53:34.686995 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.680694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43631 (local address 127.8.57.1:39013)
0929 02:53:34.681576 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0929 02:53:34.681598 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:53:34.681719 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:53:34.682003 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:53:34.682007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:53:34.682043 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:53:34.682437 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.682445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.683357 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.683360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.684821 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:53:34.684833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.685913 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.685920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:53:34.686062 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.686084 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:53:34.686172 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 02:53:34.686284 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":39,"thread_start_us":483,"threads_started":1}
I20260929 02:53:34.687497 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e967ff781ad40a9adb83c88f03c0b86" candidate_uuid: "e08d42db15be49339719d32f8d109504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" is_pre_election: true
I20260929 02:53:34.687896 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e967ff781ad40a9adb83c88f03c0b86" candidate_uuid: "e08d42db15be49339719d32f8d109504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344b76a2a3ac4148881553b1856c0051" is_pre_election: true
I20260929 02:53:34.688079 19155 raft_consensus.cc:2454] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08d42db15be49339719d32f8d109504 in term 0.
I20260929 02:53:34.688355 19022 raft_consensus.cc:2454] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08d42db15be49339719d32f8d109504 in term 0.
I20260929 02:53:34.688683 18821 leader_election.cc:302] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5acd03e4f8c0416ea94cef36dcd678e1, e08d42db15be49339719d32f8d109504; no voters:
I20260929 02:53:34.689049 19226 raft_consensus.cc:2781] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:53:34.689167 19226 raft_consensus.cc:482] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:53:34.689260 19226 raft_consensus.cc:3037] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:34.691278 19226 raft_consensus.cc:504] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.691782 19226 leader_election.cc:288] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [CANDIDATE]: Term 1 election: Requested vote from peers 5acd03e4f8c0416ea94cef36dcd678e1 (127.8.57.3:33893), 344b76a2a3ac4148881553b1856c0051 (127.8.57.2:43631)
I20260929 02:53:34.692247 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e967ff781ad40a9adb83c88f03c0b86" candidate_uuid: "e08d42db15be49339719d32f8d109504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344b76a2a3ac4148881553b1856c0051"
I20260929 02:53:34.692287 19155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e967ff781ad40a9adb83c88f03c0b86" candidate_uuid: "e08d42db15be49339719d32f8d109504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acd03e4f8c0416ea94cef36dcd678e1"
I20260929 02:53:34.692447 19022 raft_consensus.cc:3037] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:34.692443 19155 raft_consensus.cc:3037] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:53:34.695427 19155 raft_consensus.cc:2454] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08d42db15be49339719d32f8d109504 in term 1.
I20260929 02:53:34.695436 19022 raft_consensus.cc:2454] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08d42db15be49339719d32f8d109504 in term 1.
I20260929 02:53:34.695799 18822 leader_election.cc:302] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344b76a2a3ac4148881553b1856c0051, e08d42db15be49339719d32f8d109504; no voters:
I20260929 02:53:34.696097 19226 raft_consensus.cc:2781] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:53:34.696458 19226 raft_consensus.cc:686] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 1 LEADER]: Becoming Leader. State: Replica: e08d42db15be49339719d32f8d109504, State: Running, Role: LEADER
I20260929 02:53:34.696877 19226 consensus_queue.cc:237] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } }
I20260929 02:53:34.700639 18744 catalog_manager.cc:5495] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 reported cstate change: term changed from 0 to 1, leader changed from <none> to e08d42db15be49339719d32f8d109504 (127.8.57.1). New cstate: current_term: 1 leader_uuid: "e08d42db15be49339719d32f8d109504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e08d42db15be49339719d32f8d109504" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37923 } health_report { overall_health: HEALTHY } } }
I20260929 02:53:34.724817 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.720267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.8.57.1:37923)
0929 02:53:34.720434 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.720440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.720457 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:53:34.720655 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.720659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.720744 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.720862 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:53:34.721154 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.721945 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.723120 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.723496 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.723577 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.723957 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:53:34.724416 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:53:34.724509 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.724580 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.724634 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 02:53:34.732402 19155 raft_consensus.cc:1261] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 1 FOLLOWER]: Refusing update from remote peer e08d42db15be49339719d32f8d109504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:34.732403 19022 raft_consensus.cc:1261] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 1 FOLLOWER]: Refusing update from remote peer e08d42db15be49339719d32f8d109504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:53:34.733371 19229 consensus_queue.cc:1035] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5acd03e4f8c0416ea94cef36dcd678e1" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:34.733675 19226 consensus_queue.cc:1035] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "344b76a2a3ac4148881553b1856c0051" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:53:34.743322 19232 mvcc.cc:205] Tried to move back new op lower bound from 7334504098733596672 to 7334504098601517056. Current Snapshot: MvccSnapshot[applied={T|T < 7334504098733596672}]
I20260929 02:53:34.749933 19234 mvcc.cc:205] Tried to move back new op lower bound from 7334504098733596672 to 7334504098601517056. Current Snapshot: MvccSnapshot[applied={T|T < 7334504098733596672}]
I20260929 02:53:34.750216 19233 mvcc.cc:205] Tried to move back new op lower bound from 7334504098733596672 to 7334504098601517056. Current Snapshot: MvccSnapshot[applied={T|T < 7334504098733596672}]
I20260929 02:53:34.757164 18744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:34.757433 18744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:53:34.761157 18744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a61b0d49f67d4e09b64044518e8e8582: Sending DeleteTablet for 3 replicas of tablet 7e967ff781ad40a9adb83c88f03c0b86
I20260929 02:53:34.762184 19135 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e967ff781ad40a9adb83c88f03c0b86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:34 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:33322
I20260929 02:53:34.762374 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e967ff781ad40a9adb83c88f03c0b86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:34 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:46194
I20260929 02:53:34.763437 18868 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e967ff781ad40a9adb83c88f03c0b86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:53:34 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:40610
I20260929 02:53:34.766863 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18805
I20260929 02:53:34.773424 19243 tablet_replica.cc:330] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: stopping tablet replica
I20260929 02:53:34.774416 19243 raft_consensus.cc:2229] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:53:34.775619 19243 raft_consensus.cc:2258] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:34.776333 19242 tablet_replica.cc:330] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: stopping tablet replica
I20260929 02:53:34.776678 19242 raft_consensus.cc:2229] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:34.777138 19242 raft_consensus.cc:2258] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:34.779062 19242 ts_tablet_manager.cc:1905] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:34.780359 19241 tablet_replica.cc:330] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: stopping tablet replica
I20260929 02:53:34.780716 19241 raft_consensus.cc:2229] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:53:34.781119 19241 raft_consensus.cc:2258] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:53:34.782298 19243 ts_tablet_manager.cc:1905] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:34.786821 19242 ts_tablet_manager.cc:1918] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Sep 29 02:53:34 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:53:34.788177 19241 ts_tablet_manager.cc:1905] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:53:34.788422 19242 log.cc:1196] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/7e967ff781ad40a9adb83c88f03c0b86
I20260929 02:53:34.788964 19242 ts_tablet_manager.cc:1939] T 7e967ff781ad40a9adb83c88f03c0b86 P 344b76a2a3ac4148881553b1856c0051: Deleting consensus metadata
I20260929 02:53:34.789716 19243 ts_tablet_manager.cc:1918] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:34.789911 19243 log.cc:1196] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/7e967ff781ad40a9adb83c88f03c0b86
I20260929 02:53:34.790479 19243 ts_tablet_manager.cc:1939] T 7e967ff781ad40a9adb83c88f03c0b86 P e08d42db15be49339719d32f8d109504: Deleting consensus metadata
I20260929 02:53:34.792157 18728 catalog_manager.cc:4841] TS e08d42db15be49339719d32f8d109504 (127.8.57.1:37923): tablet 7e967ff781ad40a9adb83c88f03c0b86 (table test-table [id=e328a1bb80fd438980a50844684afcf8]) successfully deleted
I20260929 02:53:34.793095 18729 catalog_manager.cc:4841] TS 344b76a2a3ac4148881553b1856c0051 (127.8.57.2:43631): tablet 7e967ff781ad40a9adb83c88f03c0b86 (table test-table [id=e328a1bb80fd438980a50844684afcf8]) successfully deleted
I20260929 02:53:34.795753 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.768371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.8.57.1:37923)
0929 02:53:34.768528 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.768534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.777618 (+ 9084us) server_negotiation.cc:407] Connection header received
0929 02:53:34.778045 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.778051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.778131 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.778201 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:34.780006 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.781126 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.782825 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.783444 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.783537 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.788534 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:34.788548 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:34.788563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:34.788610 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:34.792494 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.793271 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.793279 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.793283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.793492 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.793942 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.793947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.793950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.794352 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:34.794631 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.795173 (+ 542us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.795403 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:53:34.797102 19241 ts_tablet_manager.cc:1918] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:53:34.797118 18928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:34.797227 19241 log.cc:1196] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/7e967ff781ad40a9adb83c88f03c0b86
I20260929 02:53:34.797626 19241 ts_tablet_manager.cc:1939] T 7e967ff781ad40a9adb83c88f03c0b86 P 5acd03e4f8c0416ea94cef36dcd678e1: Deleting consensus metadata
I20260929 02:53:34.799021 18728 catalog_manager.cc:4841] TS 5acd03e4f8c0416ea94cef36dcd678e1 (127.8.57.3:33893): tablet 7e967ff781ad40a9adb83c88f03c0b86 (table test-table [id=e328a1bb80fd438980a50844684afcf8]) successfully deleted
I20260929 02:53:34.891710 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18805
I20260929 02:53:34.905931 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18939
Sep 29 02:53:34 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:53:34.919566 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:34.906615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46196 (local address 127.8.57.2:43631)
0929 02:53:34.906789 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:34.906794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:34.906812 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:53:34.906960 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:34.906963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:34.907025 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:34.907102 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:34.908279 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.909005 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.910207 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:34.910602 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:34.910690 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:34.914605 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:34.914618 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:34.914637 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:34.914669 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:34.917085 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.917712 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.917722 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.917726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.917864 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:34.918222 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:34.918227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:34.918246 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:34.918641 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:34.918844 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:34.919099 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 02:53:34.919245 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:53:34.920148 19062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:35.006146 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18939
I20260929 02:53:35.018915 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 19072
Sep 29 02:53:35 dist-test-slave-1dkt krb5kdc[18699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:53:35.032505 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:53:35.019597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.8.57.3:33893)
0929 02:53:35.019762 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:53:35.019768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:53:35.019787 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:53:35.019971 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:53:35.019977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:53:35.020045 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:53:35.020138 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:53:35.021298 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:35.022001 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:35.023213 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:53:35.023602 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:53:35.023703 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:53:35.027539 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:53:35.027552 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:53:35.027567 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:53:35.027604 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:53:35.029997 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:35.030579 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:35.030587 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:35.030589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:35.030719 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:53:35.031047 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:53:35.031050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:53:35.031052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:53:35.031394 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:53:35.031647 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0929 02:53:35.031954 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 02:53:35.032158 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:53:35.033078 19195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:35.118793 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 19072
I20260929 02:53:35.131675 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 18712
I20260929 02:53:35.133024 18774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:53:35.242453 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 18712
2026-09-29T02:53:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3889 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:53:35 dist-test-slave-1dkt krb5kdc[19251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:53:35 dist-test-slave-1dkt krb5kdc[19251](info): set up 2 sockets
Sep 29 02:53:35 dist-test-slave-1dkt krb5kdc[19251](info): commencing operation
krb5kdc: starting...
W20260929 02:53:36.244620 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:53:36 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:53:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:53:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/rangeradmin_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/rangeradmin_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/rangerlookup_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/rangerlookup_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
I20260929 02:53:36.534894 8420 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskV6jRaD/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/postgres -l logfile start
2026-09-29 02:53:38.816 UTC [19281] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 02:53:38.816 UTC [19281] LOG: listening on IPv4 address "127.8.57.20", port 37617
2026-09-29 02:53:38.820 UTC [19281] LOG: listening on Unix socket "/tmp/.s.PGSQL.37617"
2026-09-29 02:53:38.841 UTC [19284] LOG: database system was shut down at 2026-09-29 02:53:37 UTC
2026-09-29 02:53:38.844 UTC [19281] LOG: database system is ready to accept connections
I20260929 02:53:39.732304 8420 mini_postgres.cc:96] Postgres bound to 37617
2026-09-29 02:53:39.747 UTC [19292] FATAL: database "slave" does not exist
127.8.57.20:37617 - accepting connections
I20260929 02:53:39.748409 8420 mini_ranger.cc:161] Starting Ranger...
I20260929 02:53:39.771643 8420 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 02:53:40.202821 8420 mini_ranger.cc:87] Created ranger Postgres database
I20260929 02:53:40.202950 8420 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-admin
2026-09-29 02:53:40,483 [I] DB FLAVOR :POSTGRES
2026-09-29 02:53:40,484 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:53:40,484 [I] Checking connection..
2026-09-29 02:53:40,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 02:53:40,730 [I] Checking connection passed.
2026-09-29 02:53:40,730 [I] --------- Verifying version history table ---------
2026-09-29 02:53:40,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:53:40,985 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:53:40,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:53:41,229 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:53:41,230 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 02:53:41,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 02:53:41.480 UTC [19356] WARNING: there is no transaction in progress
2026-09-29 02:53:41,486 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 02:53:41,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:53:41,719 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 02:53:41,719 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:53:41,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 02:53:41,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 02:53:42,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1dkt.c.gcp-upstream.internal','N') ;"
2026-09-29 02:53:42,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:53:42,686 [I] Table x_portal_user does not exist in database ranger
2026-09-29 02:53:42,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:53:42,935 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 02:53:42,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:53:43,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:53:43,387 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 02:53:43,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 02:53:44.324 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.339 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.355 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.372 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.388 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.404 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.457 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.464 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.479 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.489 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.499 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.512 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.522 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.528 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.533 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:44.539 UTC [19464] WARNING: there is no transaction in progress
2026-09-29 02:53:45,205 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 02:53:45,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 02:53:45,452 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 02:53:45,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:53:45,690 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 02:53:45,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:53:45,953 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 02:53:45,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:53:46,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:53:46,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1dkt.c.gcp-upstream.internal';"
2026-09-29 02:53:46,682 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 02:53:46,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 02:53:46,927 [I] DB_PATCHES have already been applied
I20260929 02:53:46.933588 8420 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-admin:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskV6jRaD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 02:53:46.933679 8420 mini_ranger.cc:193] Using host: 127.8.57.20
I20260929 02:53:46.947227 8420 mini_ranger.cc:227] Ranger admin URL: http://127.8.57.20:45525
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 2:53:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 02:53:47.950933 8420 mini_ranger.cc:160] Time spent starting Ranger: real 8.203s user 0.000s sys 0.035s
Sep 29, 2026 2:54:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 2:54:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 2:54:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 2:54:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 2:54:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 02:54:09.367815 8420 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/rangerkms_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/rangerkms_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.57.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.57.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 02:54:09 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650449, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 02:54:09.524008 8420 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 02:54:09.545625 8420 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 02:54:10.369798 8420 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 02:54:10.369973 8420 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms
I20260929 02:54:10.370117 8420 mini_ranger_kms.cc:227] Using postgres at 127.8.57.20:37617
2026-09-29 02:54:10,784 [I] DB FLAVOR :POSTGRES
2026-09-29 02:54:10,785 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:54:10,785 [I] Checking connection
2026-09-29 02:54:10,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 02:54:11,024 [I] connection success
2026-09-29 02:54:11,024 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 02:54:11,024 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 02:54:11,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 02:54:11,254 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 02:54:11,255 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:54:11,255 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 02:54:11,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:37617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 02:54:11,497 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 02:54:11.926584 8420 mini_ranger_kms.cc:310] Created kms service
I20260929 02:54:12.099038 8420 mini_ranger_kms.cc:326] Created kudu user
I20260929 02:54:12.140425 8420 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 02:54:12 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650449, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:54:12.527808 8420 mini_ranger_kms.cc:384] Added ranger policy
I20260929 02:54:12.528141 8420 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms:/tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 02:54:12.528184 8420 mini_ranger_kms.cc:240] Using host: 127.8.57.20
I20260929 02:54:12.548134 8420 mini_ranger_kms.cc:276] Ranger KMS PID: 19644
I20260929 02:54:12.548295 8420 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.57.20:42683
Sep 29, 2026 2:54:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:54:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:54:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 2:54:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 2:54:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 2:54:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 02:54:14.656070 8420 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.132s user 0.125s sys 0.311s
I20260929 02:54:14.656306 8420 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 02:54:14.656494 8420 mini_ranger_kms.cc:401] 127.8.57.20:42683/kms/v1/keys
Sep 29 02:54:14 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650454, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.57.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 02:54:15 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650454, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.57.20@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
Sep 29 02:54:15 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650454, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.57.20@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
Sep 29 02:54:17 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:54:17.928090 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36203
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:36203
--ranger_config_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:42683/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:54:18.261818 19695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:54:18.262099 19695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:54:18.262180 19695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:54:18.269500 19695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:54:18.269616 19695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:54:18.269675 19695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:54:18.269752 19695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:54:18.269801 19695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:54:18.280323 19695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.57.20:42683/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:36203
--ranger_config_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:54:18.281543 19695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:54:18.282774 19695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:54:18.294217 19700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:54:18.294291 19701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:54:18.294256 19703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:54:18.295099 19695 server_base.cc:1034] running on GCE node
I20260929 02:54:18.295812 19695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:54:18.297215 19695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:54:18.298408 19695 hybrid_clock.cc:648] HybridClock initialized: now 1790650458298370 us; error 30 us; skew 500 ppm
I20260929 02:54:18.298750 19695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:54:18 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650458, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:54:18.301740 19695 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:54:18.303131 19695 webserver.cc:460] Webserver started at http://127.8.57.62:37081/ using document root <none> and password file <none>
I20260929 02:54:18.303737 19695 fs_manager.cc:359] Metadata directory not provided
I20260929 02:54:18.303846 19695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:54:18.304148 19695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:54:18 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650458, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:54:18.359102 19695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "018887d3fc9543c2b3d2a6d5c7b61022"
format_stamp: "Formatted at 2026-09-29 02:54:18 on dist-test-slave-1dkt"
server_key: "6152eff8fdf81c59c9f0434ef68d41bc"
server_key_iv: "33ce8f77e664907f1fbc63d752889ab7"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:18.359922 19695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "018887d3fc9543c2b3d2a6d5c7b61022"
format_stamp: "Formatted at 2026-09-29 02:54:18 on dist-test-slave-1dkt"
server_key: "6152eff8fdf81c59c9f0434ef68d41bc"
server_key_iv: "33ce8f77e664907f1fbc63d752889ab7"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:18.365501 19695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:54:18.381784 19710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:18.383585 19695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260929 02:54:18.383782 19695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "018887d3fc9543c2b3d2a6d5c7b61022"
format_stamp: "Formatted at 2026-09-29 02:54:18 on dist-test-slave-1dkt"
server_key: "6152eff8fdf81c59c9f0434ef68d41bc"
server_key_iv: "33ce8f77e664907f1fbc63d752889ab7"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:18.384065 19695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:54:18.409221 19695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:54:18.413002 19695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:54:18.413262 19695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:54:18.432207 19695 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:36203
I20260929 02:54:18.432191 19761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:36203 every 8 connection(s)
I20260929 02:54:18.433954 19695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:54:18.437775 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:54:18.443055 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 19695
I20260929 02:54:18.443257 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:54:18.444070 8420 external_mini_cluster.cc:1428] Setting key 4b78c5d2d7d23673e3da6964dca76b96
I20260929 02:54:18.449170 19762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022: Bootstrap starting.
I20260929 02:54:18.452711 19762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022: Neither blocks nor log segments found. Creating new log.
I20260929 02:54:18.454087 19762 log.cc:824] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022: Log is configured to *not* fsync() on all Append() calls
I20260929 02:54:18.456835 19762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022: No bootstrap required, opened a new log
Sep 29 02:54:18 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:54:18.464401 19762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36203 } }
I20260929 02:54:18.464730 19762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:54:18.464825 19762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018887d3fc9543c2b3d2a6d5c7b61022, State: Initialized, Role: FOLLOWER
I20260929 02:54:18.464964 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:18.446154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.8.57.62:36203)
0929 02:54:18.446929 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:18.446940 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:18.446993 (+ 53us) server_negotiation.cc:407] Connection header received
0929 02:54:18.448641 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:18.448663 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:18.449053 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:18.449385 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:18.450839 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:18.452271 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:18.453605 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:18.454276 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:18.454438 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:18.458238 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:18.458246 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:18.458263 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:18.458301 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:18.461237 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:18.461852 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:18.461860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:18.461865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:18.462009 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:18.462345 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:18.462350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:18.462353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:18.462861 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:18.463137 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:18.463551 (+ 414us) server_negotiation.cc:299] Negotiation successful
0929 02:54:18.463959 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":371,"threads_started":1}
I20260929 02:54:18.465646 19762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36203 } }
I20260929 02:54:18.465931 19762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:54:18.466063 19762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:54:18.466220 19762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:54:18.468608 19762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36203 } }
I20260929 02:54:18.469049 19762 leader_election.cc:302] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 018887d3fc9543c2b3d2a6d5c7b61022; no voters:
I20260929 02:54:18.469949 19762 leader_election.cc:288] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:54:18.470063 19767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:54:18.470625 19767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [term 1 LEADER]: Becoming Leader. State: Replica: 018887d3fc9543c2b3d2a6d5c7b61022, State: Running, Role: LEADER
I20260929 02:54:18.471091 19767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36203 } }
I20260929 02:54:18.471933 19762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:54:18.473601 19769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 018887d3fc9543c2b3d2a6d5c7b61022. Latest consensus state: current_term: 1 leader_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36203 } } }
I20260929 02:54:18.473577 19768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018887d3fc9543c2b3d2a6d5c7b61022" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36203 } } }
I20260929 02:54:18.473970 19769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [sys.catalog]: This master's current role is: LEADER
I20260929 02:54:18.473970 19768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022 [sys.catalog]: This master's current role is: LEADER
I20260929 02:54:18.475325 19772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:54:18.481004 19772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:54:18.481634 19762 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 02:54:18.503798 19772 catalog_manager.cc:1347] Generated new cluster ID: 465bec6d73b2449bb713deddc575a704
I20260929 02:54:18.503978 19772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:54:18.564139 19772 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:54:18.566022 19772 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:54:18.575978 19772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022: Generated new TSK 0
I20260929 02:54:18.577019 19772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:54:19.231958 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:42683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36203
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:54:19 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 02:54:19.657393 19807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:54:19.657794 19807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:54:19.657949 19807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:54:19.669777 19807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:54:19.669919 19807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:54:19.670109 19807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:54:19.685197 19807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.57.20:42683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:54:19.686875 19807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:54:19.688614 19807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:54:19.705018 19817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:54:19.705458 19818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:54:19.707653 19807 server_base.cc:1034] running on GCE node
W20260929 02:54:19.708060 19821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:54:19.708791 19807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:54:19.709946 19807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:54:19.711207 19807 hybrid_clock.cc:648] HybridClock initialized: now 1790650459711147 us; error 45 us; skew 500 ppm
I20260929 02:54:19.711589 19807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:54:19 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:54:19.715629 19807 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:54:19.717933 19807 webserver.cc:460] Webserver started at http://127.8.57.1:35479/ using document root <none> and password file <none>
I20260929 02:54:19.718871 19807 fs_manager.cc:359] Metadata directory not provided
I20260929 02:54:19.719015 19807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:54:19.719363 19807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:54:19 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:54:19.744700 19807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "5bc60f9500a04da483e2bcc213831162"
format_stamp: "Formatted at 2026-09-29 02:54:19 on dist-test-slave-1dkt"
server_key: "59b6b755e2f94b337eafc8d2b5c5d8f7"
server_key_iv: "0f99a92a71f06ad5be58ac7da3dc514f"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:19.745832 19807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "5bc60f9500a04da483e2bcc213831162"
format_stamp: "Formatted at 2026-09-29 02:54:19 on dist-test-slave-1dkt"
server_key: "59b6b755e2f94b337eafc8d2b5c5d8f7"
server_key_iv: "0f99a92a71f06ad5be58ac7da3dc514f"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:19.773397 19807 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.015s sys 0.013s
I20260929 02:54:19.800431 19827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:19.801987 19807 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.005s
I20260929 02:54:19.802151 19807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "5bc60f9500a04da483e2bcc213831162"
format_stamp: "Formatted at 2026-09-29 02:54:19 on dist-test-slave-1dkt"
server_key: "59b6b755e2f94b337eafc8d2b5c5d8f7"
server_key_iv: "0f99a92a71f06ad5be58ac7da3dc514f"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:19.802356 19807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:54:19.819128 19807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:54:19.826619 19807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:54:19.826957 19807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:54:19.832417 19807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:54:19.836257 19807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:54:19.836354 19807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:19.836481 19807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:54:19.836544 19807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:19.878698 19807 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:40633
I20260929 02:54:19.878720 19939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:40633 every 8 connection(s)
I20260929 02:54:19.880326 19807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:54:19.883387 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 19807
I20260929 02:54:19.883610 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:54:19.884456 8420 external_mini_cluster.cc:1428] Setting key 739c9d7fc8d361195485e2f89feff2dd
Sep 29 02:54:19 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:54:19.941408 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:19.895810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:46125 (local address 127.8.57.62:36203)
0929 02:54:19.902753 (+ 6943us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:19.902760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:19.904021 (+ 1261us) server_negotiation.cc:407] Connection header received
0929 02:54:19.913142 (+ 9121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:19.913151 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:19.913283 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:19.913398 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:19.923013 (+ 9615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:19.924097 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:19.928409 (+ 4312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:19.929023 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:19.929113 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:19.933518 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:19.933526 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:19.933531 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:19.933584 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:19.937044 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:19.938052 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:19.938057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:19.938060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:19.938196 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:19.939057 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:19.939061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:19.939063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:19.939334 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:19.939490 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:19.940828 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0929 02:54:19.941049 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6803,"thread_start_us":6400,"threads_started":1}
I20260929 02:54:19.943735 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:19.895129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36203 (local address 127.8.57.1:46125)
0929 02:54:19.902937 (+ 7808us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:19.903005 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:19.910012 (+ 7007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:19.914271 (+ 4259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:19.914286 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:19.918772 (+ 4486us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:54:19.919791 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:19.919822 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:19.925176 (+ 5354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:19.925181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:19.927619 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:19.927648 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:19.929435 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:19.929443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:19.929716 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:19.930494 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:54:19.930524 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:54:19.933238 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:54:19.937468 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:54:19.937479 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:54:19.937498 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:54:19.937893 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:54:19.938316 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:54:19.938320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:54:19.938322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:54:19.938571 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:54:19.939978 (+ 1407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:54:19.939985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:54:19.940412 (+ 427us) client_negotiation.cc:769] Sending connection context
0929 02:54:19.941858 (+ 1446us) client_negotiation.cc:240] Negotiation successful
0929 02:54:19.942384 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6092,"thread_start_us":4962,"threads_started":1}
I20260929 02:54:19.945935 19940 heartbeater.cc:347] Connected to a master server at 127.8.57.62:36203
I20260929 02:54:19.946255 19940 heartbeater.cc:464] Registering TS with master...
I20260929 02:54:19.947139 19940 heartbeater.cc:511] Master 127.8.57.62:36203 requested a full tablet report, sending...
I20260929 02:54:19.949622 19727 ts_manager.cc:194] Registered new tserver with Master: 5bc60f9500a04da483e2bcc213831162 (127.8.57.1:40633)
I20260929 02:54:19.951620 19727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:46125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:54:19.979933 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:42683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36203
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:54:19 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
Sep 29 02:54:20 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
W20260929 02:54:20.523289 19946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:54:20.523797 19946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:54:20.524025 19946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:54:20.535308 19946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:54:20.535540 19946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:54:20.535785 19946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:54:20.551792 19946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.57.20:42683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:54:20.553748 19946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:54:20.555546 19946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:54:20.604101 19954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:54:20.604607 19946 server_base.cc:1034] running on GCE node
W20260929 02:54:20.605167 19957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:54:20.605439 19953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:54:20.606724 19946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:54:20.613428 19946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:54:20.614715 19946 hybrid_clock.cc:648] HybridClock initialized: now 1790650460614612 us; error 103 us; skew 500 ppm
I20260929 02:54:20.615187 19946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:54:20 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650460, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:54:20.619446 19946 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:54:20.639919 19946 webserver.cc:460] Webserver started at http://127.8.57.2:45923/ using document root <none> and password file <none>
I20260929 02:54:20.641084 19946 fs_manager.cc:359] Metadata directory not provided
I20260929 02:54:20.641328 19946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:54:20.641782 19946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:54:20 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650460, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:54:20.666504 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "97ce53beaa8d470abc45316d7060fffa"
format_stamp: "Formatted at 2026-09-29 02:54:20 on dist-test-slave-1dkt"
server_key: "7fd6c51b829e85398010f94d636a5bf0"
server_key_iv: "09da0ac2888bce5a68bcae007baed32d"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:20.668129 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "97ce53beaa8d470abc45316d7060fffa"
format_stamp: "Formatted at 2026-09-29 02:54:20 on dist-test-slave-1dkt"
server_key: "7fd6c51b829e85398010f94d636a5bf0"
server_key_iv: "09da0ac2888bce5a68bcae007baed32d"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:20.686255 19946 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.004s sys 0.012s
I20260929 02:54:20.726495 19965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:20.728601 19946 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.005s sys 0.017s
I20260929 02:54:20.728853 19946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "97ce53beaa8d470abc45316d7060fffa"
format_stamp: "Formatted at 2026-09-29 02:54:20 on dist-test-slave-1dkt"
server_key: "7fd6c51b829e85398010f94d636a5bf0"
server_key_iv: "09da0ac2888bce5a68bcae007baed32d"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:20.729203 19946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:54:20.752315 19946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:54:20.756582 19946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:54:20.756846 19946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:54:20.757851 19946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:54:20.759203 19946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:54:20.759277 19946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:20.759395 19946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:54:20.759446 19946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:20.802345 19946 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:40837
I20260929 02:54:20.802347 20082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:40837 every 8 connection(s)
I20260929 02:54:20.803615 19946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:54:20.809413 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 19946
I20260929 02:54:20.809597 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:54:20.810144 8420 external_mini_cluster.cc:1428] Setting key 55fcef31a8b4af13aa3ad367494071da
Sep 29 02:54:20 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650460, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:54:20.828768 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:20.807605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:34555 (local address 127.8.57.62:36203)
0929 02:54:20.812238 (+ 4633us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:20.812244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:20.812258 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:54:20.812303 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:20.812308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:20.812379 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:20.812474 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:20.814200 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:20.815795 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:20.817965 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:20.818577 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:20.818697 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:20.822062 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:20.822068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:20.822072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:20.822109 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:20.825309 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:20.826172 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:20.826179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:20.826182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:20.826382 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:20.826855 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:20.826860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:20.826864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:20.827203 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:20.827343 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:20.828168 (+ 825us) server_negotiation.cc:299] Negotiation successful
0929 02:54:20.828388 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4527,"thread_start_us":4415,"threads_started":1}
I20260929 02:54:20.829941 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:20.807025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36203 (local address 127.8.57.2:34555)
0929 02:54:20.807826 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:20.807910 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:20.810413 (+ 2503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:20.812738 (+ 2325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:20.812748 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:20.813239 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:54:20.814013 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:20.814040 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:20.815980 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:20.815986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:20.817809 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:20.817824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:20.818790 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:20.818797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:20.819036 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:20.819747 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:54:20.819773 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:54:20.821859 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:54:20.825452 (+ 3593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:54:20.825461 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:54:20.825478 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:54:20.826027 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:54:20.826527 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:54:20.826531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:54:20.826534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:54:20.826751 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:54:20.827345 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:54:20.827353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:54:20.827747 (+ 394us) client_negotiation.cc:769] Sending connection context
0929 02:54:20.828080 (+ 333us) client_negotiation.cc:240] Negotiation successful
0929 02:54:20.828616 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":355,"threads_started":1}
I20260929 02:54:20.831795 20083 heartbeater.cc:347] Connected to a master server at 127.8.57.62:36203
I20260929 02:54:20.832154 20083 heartbeater.cc:464] Registering TS with master...
I20260929 02:54:20.832989 20083 heartbeater.cc:511] Master 127.8.57.62:36203 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:54:20.834684 19727 ts_manager.cc:194] Registered new tserver with Master: 97ce53beaa8d470abc45316d7060fffa (127.8.57.2:40837)
I20260929 02:54:20.835775 19727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:34555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:54:20.864408 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:42683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36203
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:54:20.955230 19940 heartbeater.cc:503] Master 127.8.57.62:36203 was elected leader, sending a full tablet report...
W20260929 02:54:21.192104 20089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:54:21.192381 20089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:54:21.192509 20089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:54:21.199694 20089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:54:21.199811 20089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:54:21.199981 20089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:54:21.210134 20089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.57.20:42683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:54:21.211398 20089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:54:21.212697 20089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:54:21.225566 20095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:54:21.225596 20097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:54:21.225620 20094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:54:21.226236 20089 server_base.cc:1034] running on GCE node
I20260929 02:54:21.226814 20089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:54:21.227669 20089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:54:21.228852 20089 hybrid_clock.cc:648] HybridClock initialized: now 1790650461228780 us; error 65 us; skew 500 ppm
I20260929 02:54:21.229120 20089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:54:21 dist-test-slave-1dkt krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650461, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:54:21.235064 20089 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:54:21.236599 20089 webserver.cc:460] Webserver started at http://127.8.57.3:33127/ using document root <none> and password file <none>
I20260929 02:54:21.237673 20089 fs_manager.cc:359] Metadata directory not provided
I20260929 02:54:21.237789 20089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:54:21.238134 20089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:54:21 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650461, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:54:21.253979 20089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "745cfd46a15b4676bee4730153c95f09"
format_stamp: "Formatted at 2026-09-29 02:54:21 on dist-test-slave-1dkt"
server_key: "d418b1071cd5b83219d6e4585d47cd72"
server_key_iv: "d024ad5c5e896de315453e076c326bb7"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:21.254810 20089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "745cfd46a15b4676bee4730153c95f09"
format_stamp: "Formatted at 2026-09-29 02:54:21 on dist-test-slave-1dkt"
server_key: "d418b1071cd5b83219d6e4585d47cd72"
server_key_iv: "d024ad5c5e896de315453e076c326bb7"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:21.260243 20089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260929 02:54:21.274838 20104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:21.276568 20089 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 02:54:21.276754 20089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "745cfd46a15b4676bee4730153c95f09"
format_stamp: "Formatted at 2026-09-29 02:54:21 on dist-test-slave-1dkt"
server_key: "d418b1071cd5b83219d6e4585d47cd72"
server_key_iv: "d024ad5c5e896de315453e076c326bb7"
server_key_version: "kuduclusterkey@0"
I20260929 02:54:21.276970 20089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:54:21.307121 20089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:54:21.311194 20089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:54:21.311483 20089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:54:21.312556 20089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:54:21.314008 20089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:54:21.314087 20089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:21.314211 20089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:54:21.314262 20089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:54:21.352928 20089 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:41045
I20260929 02:54:21.352938 20216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:41045 every 8 connection(s)
I20260929 02:54:21.354156 20089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:54:21.355556 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 20089
I20260929 02:54:21.355723 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:54:21.356348 8420 external_mini_cluster.cc:1428] Setting key fe329b2d36ff921833fcce72776de758
Sep 29 02:54:21 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650461, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:54:21.385725 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.357446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36203 (local address 127.8.57.3:44255)
0929 02:54:21.365752 (+ 8306us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:21.365810 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:21.368356 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:21.369203 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:21.369217 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:21.369868 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:54:21.370738 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.370765 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.372234 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.372238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.373561 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.373580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.374533 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.374537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.374737 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.375366 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:54:21.375389 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:54:21.377619 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:54:21.381327 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:54:21.381335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:54:21.381353 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:54:21.381788 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:54:21.382203 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:54:21.382212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:54:21.382215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:54:21.382445 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:54:21.383037 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:54:21.383044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:54:21.383459 (+ 415us) client_negotiation.cc:769] Sending connection context
0929 02:54:21.383792 (+ 333us) client_negotiation.cc:240] Negotiation successful
0929 02:54:21.384351 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7981,"thread_start_us":7732,"threads_started":1}
I20260929 02:54:21.386834 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.356737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:44255 (local address 127.8.57.62:36203)
0929 02:54:21.365141 (+ 8404us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.365147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.365963 (+ 816us) server_negotiation.cc:407] Connection header received
0929 02:54:21.368712 (+ 2749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.368717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.368789 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.368880 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:21.370943 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.372094 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.373710 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.374352 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.374474 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.377836 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:21.377843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:21.377848 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:21.377888 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:21.381184 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.381892 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.381897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.381901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.382070 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.382583 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.382588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.382592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.382909 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:21.383057 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.386234 (+ 3177us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.386479 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8268,"thread_start_us":8097,"threads_started":1}
I20260929 02:54:21.387845 20217 heartbeater.cc:347] Connected to a master server at 127.8.57.62:36203
I20260929 02:54:21.388159 20217 heartbeater.cc:464] Registering TS with master...
I20260929 02:54:21.388870 20217 heartbeater.cc:511] Master 127.8.57.62:36203 requested a full tablet report, sending...
I20260929 02:54:21.390250 19727 ts_manager.cc:194] Registered new tserver with Master: 745cfd46a15b4676bee4730153c95f09 (127.8.57.3:41045)
I20260929 02:54:21.391049 19727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:44255
I20260929 02:54:21.396481 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:54:21.414790 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.401001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.8.57.62:36203)
0929 02:54:21.401199 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.401206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.401567 (+ 361us) server_negotiation.cc:407] Connection header received
0929 02:54:21.401729 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.401734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.401805 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.401880 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:21.403600 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.404344 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.406404 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.406862 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.406955 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.409888 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:21.409893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:21.409896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:21.409934 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:21.412329 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.413071 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.413075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.413077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.413197 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.413551 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.413554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.413556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.413802 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:21.413977 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.414366 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.414542 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:54:21.418637 19727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:54:21.426061 19727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:54:21.456605 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.438962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.8.57.2:40837)
0929 02:54:21.439480 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.439488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.445417 (+ 5929us) server_negotiation.cc:407] Connection header received
0929 02:54:21.445802 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.445811 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.446075 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.446229 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:54:21.447136 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.449636 (+ 2500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.452433 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.454793 (+ 2360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.454893 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.455088 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.456101 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.456288 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":322,"threads_started":1}
I20260929 02:54:21.457600 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.438691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:40837 (local address 127.0.0.1:42256)
0929 02:54:21.444412 (+ 5721us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:21.444457 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:21.445560 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:21.446543 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:21.446551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:21.446587 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:54:21.446949 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.446960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.449800 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.449805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.452281 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.452294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.455604 (+ 3310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.455610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.455803 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.455868 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 02:54:21.457057 (+ 1189us) client_negotiation.cc:240] Negotiation successful
0929 02:54:21.457259 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5471,"thread_start_us":5336,"threads_started":1}
I20260929 02:54:21.461998 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.448325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:41045 (local address 127.0.0.1:57282)
0929 02:54:21.453244 (+ 4919us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:21.453267 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:21.453374 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:21.454793 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:21.454797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:21.454822 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:54:21.455618 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.455637 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.457424 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.457429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.459337 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.459351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.461309 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.461315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.461469 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.461501 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:54:21.461582 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:54:21.461716 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4796,"thread_start_us":4665,"threads_started":1}
I20260929 02:54:21.462344 20018 tablet_service.cc:1467] Processing CreateTablet for tablet 45e9cef5b3494ad384d89df195859d99 (DEFAULT_TABLE table=test-table [id=2f06294dee3346229107b934d1a6e89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:54:21.463176 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.445079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.8.57.3:41045)
0929 02:54:21.453899 (+ 8820us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.453908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.453942 (+ 34us) server_negotiation.cc:407] Connection header received
0929 02:54:21.454035 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.454043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.454322 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.454507 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:54:21.455814 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.457275 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.459512 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.460869 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.460980 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.461184 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.462719 (+ 1535us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.462868 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8653,"thread_start_us":10978,"threads_started":1}
I20260929 02:54:21.464413 20018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e9cef5b3494ad384d89df195859d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:54:21.467530 20152 tablet_service.cc:1467] Processing CreateTablet for tablet 45e9cef5b3494ad384d89df195859d99 (DEFAULT_TABLE table=test-table [id=2f06294dee3346229107b934d1a6e89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:54:21.468984 20152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e9cef5b3494ad384d89df195859d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:54:21.472864 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.458053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40633 (local address 127.0.0.1:33456)
0929 02:54:21.462449 (+ 4396us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:21.462467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:21.462557 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:21.465431 (+ 2874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:21.465435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:21.465460 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:54:21.465607 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.465616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.467404 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.467409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.469156 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.469169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.472198 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.472204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.472385 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.472421 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:54:21.472482 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 02:54:21.472615 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4296}
I20260929 02:54:21.473933 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.453565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.8.57.1:40633)
0929 02:54:21.464621 (+ 11056us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.464629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.464657 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:54:21.464750 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.464756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.464989 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.465191 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:54:21.465769 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.467262 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.469382 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.472013 (+ 2631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.473190 (+ 1177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.473414 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.473480 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.473635 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10877,"threads_started":1}
I20260929 02:54:21.477820 19875 tablet_service.cc:1467] Processing CreateTablet for tablet 45e9cef5b3494ad384d89df195859d99 (DEFAULT_TABLE table=test-table [id=2f06294dee3346229107b934d1a6e89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:54:21.479364 19875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e9cef5b3494ad384d89df195859d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:54:21.493618 20235 tablet_bootstrap.cc:492] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Bootstrap starting.
I20260929 02:54:21.494365 20236 tablet_bootstrap.cc:492] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Bootstrap starting.
I20260929 02:54:21.495412 20237 tablet_bootstrap.cc:492] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Bootstrap starting.
I20260929 02:54:21.499606 20236 tablet_bootstrap.cc:654] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Neither blocks nor log segments found. Creating new log.
I20260929 02:54:21.499605 20237 tablet_bootstrap.cc:654] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Neither blocks nor log segments found. Creating new log.
I20260929 02:54:21.499956 20235 tablet_bootstrap.cc:654] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Neither blocks nor log segments found. Creating new log.
I20260929 02:54:21.500844 20236 log.cc:824] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Log is configured to *not* fsync() on all Append() calls
I20260929 02:54:21.500844 20237 log.cc:824] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Log is configured to *not* fsync() on all Append() calls
I20260929 02:54:21.501204 20235 log.cc:824] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Log is configured to *not* fsync() on all Append() calls
I20260929 02:54:21.506373 20235 tablet_bootstrap.cc:492] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: No bootstrap required, opened a new log
I20260929 02:54:21.506960 20235 ts_tablet_manager.cc:1397] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260929 02:54:21.518967 20236 tablet_bootstrap.cc:492] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: No bootstrap required, opened a new log
I20260929 02:54:21.519385 20236 ts_tablet_manager.cc:1397] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.018s
I20260929 02:54:21.520191 20235 raft_consensus.cc:348] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.520584 20237 tablet_bootstrap.cc:492] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: No bootstrap required, opened a new log
I20260929 02:54:21.520671 20235 raft_consensus.cc:374] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:54:21.520771 20235 raft_consensus.cc:729] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ce53beaa8d470abc45316d7060fffa, State: Initialized, Role: FOLLOWER
I20260929 02:54:21.520867 20237 ts_tablet_manager.cc:1397] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.016s
I20260929 02:54:21.521514 20235 consensus_queue.cc:260] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.530206 20236 raft_consensus.cc:348] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.530726 20236 raft_consensus.cc:374] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:54:21.530854 20236 raft_consensus.cc:729] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745cfd46a15b4676bee4730153c95f09, State: Initialized, Role: FOLLOWER
I20260929 02:54:21.531613 20237 raft_consensus.cc:348] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.532088 20237 raft_consensus.cc:374] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:54:21.532196 20237 raft_consensus.cc:729] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc60f9500a04da483e2bcc213831162, State: Initialized, Role: FOLLOWER
I20260929 02:54:21.532709 20235 ts_tablet_manager.cc:1428] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Time spent starting tablet: real 0.026s user 0.013s sys 0.009s
I20260929 02:54:21.532779 20083 heartbeater.cc:503] Master 127.8.57.62:36203 was elected leader, sending a full tablet report...
I20260929 02:54:21.532774 20237 consensus_queue.cc:260] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.532333 20236 consensus_queue.cc:260] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.540132 20217 heartbeater.cc:503] Master 127.8.57.62:36203 was elected leader, sending a full tablet report...
I20260929 02:54:21.540500 20237 ts_tablet_manager.cc:1428] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Time spent starting tablet: real 0.019s user 0.018s sys 0.001s
I20260929 02:54:21.540803 20236 ts_tablet_manager.cc:1428] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Time spent starting tablet: real 0.021s user 0.006s sys 0.013s
W20260929 02:54:21.555696 20084 tablet.cc:2369] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:54:21.606556 20218 tablet.cc:2369] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:54:21.667049 20243 raft_consensus.cc:482] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:54:21.667351 20243 raft_consensus.cc:504] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.668951 20243 leader_election.cc:288] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97ce53beaa8d470abc45316d7060fffa (127.8.57.2:40837), 5bc60f9500a04da483e2bcc213831162 (127.8.57.1:40633)
W20260929 02:54:21.674355 19941 tablet.cc:2369] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:54:21.674630 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.669335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:46599 (local address 127.8.57.2:40837)
0929 02:54:21.669498 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.669504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.669567 (+ 63us) server_negotiation.cc:407] Connection header received
0929 02:54:21.669776 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.669780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.669848 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.669942 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:54:21.670387 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.671282 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.673070 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.673837 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.673958 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.674055 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.674284 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.674405 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:54:21.674921 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.669305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:40837 (local address 127.8.57.3:46599)
0929 02:54:21.669482 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:21.669501 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:21.669645 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:21.669977 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:21.669982 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:21.670011 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:54:21.670224 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.670235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.671429 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.671434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.672895 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.672906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.673986 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.673992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.674148 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.674171 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:54:21.674247 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:54:21.674369 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 02:54:21.675858 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.669993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:60635 (local address 127.8.57.1:40633)
0929 02:54:21.670203 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.670209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.670579 (+ 370us) server_negotiation.cc:407] Connection header received
0929 02:54:21.670680 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.670685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.670759 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.670849 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:54:21.671365 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.672146 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.673794 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.674895 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.675023 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.675145 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.675515 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.675646 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:54:21.675926 20038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e9cef5b3494ad384d89df195859d99" candidate_uuid: "745cfd46a15b4676bee4730153c95f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97ce53beaa8d470abc45316d7060fffa" is_pre_election: true
I20260929 02:54:21.676380 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.669716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:40633 (local address 127.8.57.3:60635)
0929 02:54:21.670484 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 02:54:21.670496 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 02:54:21.670588 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:54:21.670882 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:54:21.670885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:54:21.670907 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:54:21.671223 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.671233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.672276 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.672280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.673655 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:54:21.673666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.675255 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.675259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:54:21.675367 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.675388 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 02:54:21.675455 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:54:21.675539 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":478,"threads_started":1}
I20260929 02:54:21.676473 20038 raft_consensus.cc:2454] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 745cfd46a15b4676bee4730153c95f09 in term 0.
I20260929 02:54:21.677165 20107 leader_election.cc:302] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 745cfd46a15b4676bee4730153c95f09, 97ce53beaa8d470abc45316d7060fffa; no voters:
I20260929 02:54:21.677362 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e9cef5b3494ad384d89df195859d99" candidate_uuid: "745cfd46a15b4676bee4730153c95f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc60f9500a04da483e2bcc213831162" is_pre_election: true
I20260929 02:54:21.677526 20243 raft_consensus.cc:2781] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:54:21.677631 20243 raft_consensus.cc:482] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:54:21.677723 20243 raft_consensus.cc:3037] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:54:21.677795 19895 raft_consensus.cc:2454] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 745cfd46a15b4676bee4730153c95f09 in term 0.
I20260929 02:54:21.680102 20243 raft_consensus.cc:504] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.680675 20243 leader_election.cc:288] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [CANDIDATE]: Term 1 election: Requested vote from peers 97ce53beaa8d470abc45316d7060fffa (127.8.57.2:40837), 5bc60f9500a04da483e2bcc213831162 (127.8.57.1:40633)
I20260929 02:54:21.681056 20038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e9cef5b3494ad384d89df195859d99" candidate_uuid: "745cfd46a15b4676bee4730153c95f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97ce53beaa8d470abc45316d7060fffa"
I20260929 02:54:21.681135 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e9cef5b3494ad384d89df195859d99" candidate_uuid: "745cfd46a15b4676bee4730153c95f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc60f9500a04da483e2bcc213831162"
I20260929 02:54:21.681217 20038 raft_consensus.cc:3037] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:54:21.681324 19895 raft_consensus.cc:3037] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:54:21.683333 20038 raft_consensus.cc:2454] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 745cfd46a15b4676bee4730153c95f09 in term 1.
I20260929 02:54:21.683421 19895 raft_consensus.cc:2454] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 745cfd46a15b4676bee4730153c95f09 in term 1.
I20260929 02:54:21.683729 20107 leader_election.cc:302] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 745cfd46a15b4676bee4730153c95f09, 97ce53beaa8d470abc45316d7060fffa; no voters:
I20260929 02:54:21.684032 20243 raft_consensus.cc:2781] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:54:21.684424 20243 raft_consensus.cc:686] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 1 LEADER]: Becoming Leader. State: Replica: 745cfd46a15b4676bee4730153c95f09, State: Running, Role: LEADER
I20260929 02:54:21.684880 20243 consensus_queue.cc:237] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } }
I20260929 02:54:21.689332 19726 catalog_manager.cc:5495] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 745cfd46a15b4676bee4730153c95f09 (127.8.57.3). New cstate: current_term: 1 leader_uuid: "745cfd46a15b4676bee4730153c95f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "745cfd46a15b4676bee4730153c95f09" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 41045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 } health_report { overall_health: UNKNOWN } } }
I20260929 02:54:21.745482 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.740720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.8.57.3:41045)
0929 02:54:21.740921 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.740926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.740944 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:54:21.741033 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.741037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.741122 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.741249 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:54:21.741532 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.742214 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.743554 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.743975 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.744068 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.744532 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:54:21.745017 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:54:21.745134 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.745168 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.745229 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:54:21.753536 20038 raft_consensus.cc:1261] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 1 FOLLOWER]: Refusing update from remote peer 745cfd46a15b4676bee4730153c95f09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:54:21.753536 19895 raft_consensus.cc:1261] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 1 FOLLOWER]: Refusing update from remote peer 745cfd46a15b4676bee4730153c95f09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:54:21.754475 20245 consensus_queue.cc:1035] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97ce53beaa8d470abc45316d7060fffa" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:54:21.754832 20243 consensus_queue.cc:1035] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bc60f9500a04da483e2bcc213831162" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 40633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:54:21.778627 19727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:54:21.778911 19727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:54:21.782482 19727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 018887d3fc9543c2b3d2a6d5c7b61022: Sending DeleteTablet for 3 replicas of tablet 45e9cef5b3494ad384d89df195859d99
I20260929 02:54:21.783568 20018 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e9cef5b3494ad384d89df195859d99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:54:21 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:42256
I20260929 02:54:21.784299 19875 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e9cef5b3494ad384d89df195859d99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:54:21 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:33456
I20260929 02:54:21.784817 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 19807
I20260929 02:54:21.784968 20152 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e9cef5b3494ad384d89df195859d99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:54:21 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:57282
I20260929 02:54:21.787837 20259 tablet_replica.cc:330] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: stopping tablet replica
I20260929 02:54:21.788335 20259 raft_consensus.cc:2229] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:54:21.788901 20259 raft_consensus.cc:2258] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:54:21.791218 20259 ts_tablet_manager.cc:1905] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:54:21.798314 20259 ts_tablet_manager.cc:1918] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:54:21.798482 20259 log.cc:1196] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/45e9cef5b3494ad384d89df195859d99
I20260929 02:54:21.798935 20259 ts_tablet_manager.cc:1939] T 45e9cef5b3494ad384d89df195859d99 P 5bc60f9500a04da483e2bcc213831162: Deleting consensus metadata
I20260929 02:54:21.800832 19713 catalog_manager.cc:4841] TS 5bc60f9500a04da483e2bcc213831162 (127.8.57.1:40633): tablet 45e9cef5b3494ad384d89df195859d99 (table test-table [id=2f06294dee3346229107b934d1a6e89b]) successfully deleted
Sep 29 02:54:21 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:54:21.812837 20260 tablet_replica.cc:330] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: stopping tablet replica
I20260929 02:54:21.814803 20260 raft_consensus.cc:2229] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:54:21.815934 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.786107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.8.57.1:40633)
0929 02:54:21.786463 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.786469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.792231 (+ 5762us) server_negotiation.cc:407] Connection header received
0929 02:54:21.792499 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.792505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.792584 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.792668 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:21.794507 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.795648 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.797619 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.798282 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.798385 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.807128 (+ 8743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:21.807147 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:21.807164 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:21.807213 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:21.811035 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.812036 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.812047 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.812051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.812226 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.813069 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.813076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.813079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.813583 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:21.813873 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.815215 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.815492 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260929 02:54:21.817665 19935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:54:21.818217 20258 tablet_replica.cc:330] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: stopping tablet replica
I20260929 02:54:21.818544 20258 raft_consensus.cc:2229] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:54:21.818991 20258 raft_consensus.cc:2258] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:54:21.819253 20260 raft_consensus.cc:2258] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:54:21.821270 20258 ts_tablet_manager.cc:1905] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:54:21.822717 20260 ts_tablet_manager.cc:1905] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:54:21.828315 20258 ts_tablet_manager.cc:1918] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:54:21.828555 20258 log.cc:1196] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/45e9cef5b3494ad384d89df195859d99
I20260929 02:54:21.829160 20260 ts_tablet_manager.cc:1918] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:54:21.829319 20260 log.cc:1196] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/45e9cef5b3494ad384d89df195859d99
I20260929 02:54:21.829622 20258 ts_tablet_manager.cc:1939] T 45e9cef5b3494ad384d89df195859d99 P 97ce53beaa8d470abc45316d7060fffa: Deleting consensus metadata
I20260929 02:54:21.829758 20260 ts_tablet_manager.cc:1939] T 45e9cef5b3494ad384d89df195859d99 P 745cfd46a15b4676bee4730153c95f09: Deleting consensus metadata
I20260929 02:54:21.831336 19713 catalog_manager.cc:4841] TS 745cfd46a15b4676bee4730153c95f09 (127.8.57.3:41045): tablet 45e9cef5b3494ad384d89df195859d99 (table test-table [id=2f06294dee3346229107b934d1a6e89b]) successfully deleted
I20260929 02:54:21.831626 19713 catalog_manager.cc:4841] TS 97ce53beaa8d470abc45316d7060fffa (127.8.57.2:40837): tablet 45e9cef5b3494ad384d89df195859d99 (table test-table [id=2f06294dee3346229107b934d1a6e89b]) successfully deleted
I20260929 02:54:21.917863 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 19807
I20260929 02:54:21.931731 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 19946
Sep 29 02:54:21 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:54:21.946678 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:21.932514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.8.57.2:40837)
0929 02:54:21.932701 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:21.932707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:21.932726 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:54:21.932856 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:21.932861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:21.932938 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:21.933016 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:21.934257 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.934995 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.936349 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:21.936737 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:21.936842 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:21.941108 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:21.941123 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:21.941139 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:21.941174 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:21.943978 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.944782 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.944789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.944791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.944919 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:21.945290 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:21.945293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:21.945295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:21.945647 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:21.945841 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:21.946166 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 02:54:21.946318 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:54:21.947345 20078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:54:22.034962 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 19946
I20260929 02:54:22.048049 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 20089
Sep 29 02:54:22 dist-test-slave-1dkt krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:54:22.063416 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:54:22.048848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.8.57.3:41045)
0929 02:54:22.049048 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:54:22.049053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:54:22.049072 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:54:22.049256 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:54:22.049262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:54:22.049334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:54:22.049412 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:54:22.050652 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:22.051451 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:22.052766 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:54:22.053228 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:54:22.053338 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:54:22.057595 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:54:22.057607 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:54:22.057621 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:54:22.057661 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:54:22.060680 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:22.061428 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:22.061437 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:22.061440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:22.061579 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:54:22.061940 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:54:22.061944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:54:22.061947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:54:22.062292 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:54:22.062585 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0929 02:54:22.062865 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 02:54:22.063086 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 02:54:22.064123 20212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:54:22.183045 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 20089
I20260929 02:54:22.197614 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 19695
I20260929 02:54:22.198704 19757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:54:22.307302 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 19695
I20260929 02:54:22.325294 8420 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 02:54:22.660271 8420 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 02:54:22.660490 8420 mini_ranger.cc:66] Stopping Ranger...
I20260929 02:54:23.005657 8420 mini_ranger.cc:68] Stopped Ranger
2026-09-29 02:54:23.005 UTC [19281] LOG: received smart shutdown request
2026-09-29 02:54:23.008 UTC [19281] LOG: background worker "logical replication launcher" (PID 19290) exited with exit code 1
2026-09-29 02:54:23.009 UTC [19285] LOG: shutting down
2026-09-29 02:54:23.150 UTC [19281] LOG: database system is shut down
2026-09-29T02:54:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48013 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:54:23 dist-test-slave-1dkt krb5kdc[20274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:54:23 dist-test-slave-1dkt krb5kdc[20274](info): set up 2 sockets
Sep 29 02:54:23 dist-test-slave-1dkt krb5kdc[20274](info): commencing operation
krb5kdc: starting...
W20260929 02:54:24.256116 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:54:24 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:54:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:54:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/rangeradmin_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/rangeradmin_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/rangerlookup_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/rangerlookup_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
I20260929 02:54:24.552524 8420 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskV6jRaD/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/postgres -l logfile start
2026-09-29 02:54:26.896 UTC [20304] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 02:54:26.897 UTC [20304] LOG: listening on IPv4 address "127.8.57.20", port 54475
2026-09-29 02:54:26.900 UTC [20304] LOG: listening on Unix socket "/tmp/.s.PGSQL.54475"
2026-09-29 02:54:26.920 UTC [20307] LOG: database system was shut down at 2026-09-29 02:54:25 UTC
2026-09-29 02:54:26.923 UTC [20304] LOG: database system is ready to accept connections
I20260929 02:54:27.811714 8420 mini_postgres.cc:96] Postgres bound to 54475
2026-09-29 02:54:27.830 UTC [20315] FATAL: database "slave" does not exist
127.8.57.20:54475 - accepting connections
I20260929 02:54:27.830966 8420 mini_ranger.cc:161] Starting Ranger...
I20260929 02:54:27.853302 8420 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 02:54:28.278921 8420 mini_ranger.cc:87] Created ranger Postgres database
I20260929 02:54:28.279029 8420 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-admin
2026-09-29 02:54:28,549 [I] DB FLAVOR :POSTGRES
2026-09-29 02:54:28,550 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:54:28,550 [I] Checking connection..
2026-09-29 02:54:28,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 02:54:28,785 [I] Checking connection passed.
2026-09-29 02:54:28,785 [I] --------- Verifying version history table ---------
2026-09-29 02:54:28,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:54:29,014 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:54:29,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:54:29,250 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:54:29,250 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 02:54:29,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 02:54:29.501 UTC [20379] WARNING: there is no transaction in progress
2026-09-29 02:54:29,507 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 02:54:29,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:54:29,740 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 02:54:29,740 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:54:29,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 02:54:29,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 02:54:30,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1dkt.c.gcp-upstream.internal','N') ;"
2026-09-29 02:54:30,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:54:30,689 [I] Table x_portal_user does not exist in database ranger
2026-09-29 02:54:30,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:54:30,941 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 02:54:30,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:54:31,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:54:31,440 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 02:54:31,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 02:54:32.384 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.401 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.417 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.434 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.450 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.466 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.523 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.530 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.544 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.555 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.565 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.580 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.591 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.598 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.605 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:32.612 UTC [20487] WARNING: there is no transaction in progress
2026-09-29 02:54:33,239 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 02:54:33,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 02:54:33,490 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 02:54:33,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:54:33,744 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 02:54:33,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:54:33,982 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 02:54:33,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:54:34,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:54:34,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1dkt.c.gcp-upstream.internal';"
2026-09-29 02:54:34,731 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 02:54:34,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 02:54:34,953 [I] DB_PATCHES have already been applied
I20260929 02:54:34.958654 8420 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-admin:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskV6jRaD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 02:54:34.958737 8420 mini_ranger.cc:193] Using host: 127.8.57.20
I20260929 02:54:34.971769 8420 mini_ranger.cc:227] Ranger admin URL: http://127.8.57.20:40223
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 2:54:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 02:54:35.980939 8420 mini_ranger.cc:160] Time spent starting Ranger: real 8.150s user 0.000s sys 0.036s
Sep 29, 2026 2:54:54 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 2:54:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 2:54:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 2:54:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 2:54:55 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 02:54:55.874987 8420 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.57.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.57.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/rangerkms_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.57.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/rangerkms_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.57.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.57.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.57.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/HTTP_127.8.57.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 02:54:56 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650496, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 02:54:56.034837 8420 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 02:54:56.052899 8420 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 02:54:56.896862 8420 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 02:54:56.896984 8420 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms
I20260929 02:54:56.897022 8420 mini_ranger_kms.cc:227] Using postgres at 127.8.57.20:54475
2026-09-29 02:54:57,217 [I] DB FLAVOR :POSTGRES
2026-09-29 02:54:57,218 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:54:57,218 [I] Checking connection
2026-09-29 02:54:57,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 02:54:57,453 [I] connection success
2026-09-29 02:54:57,453 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 02:54:57,453 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 02:54:57,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 02:54:57,680 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 02:54:57,680 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:54:57,680 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 02:54:57,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.57.20:54475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 02:54:57,920 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 02:54:58.352692 8420 mini_ranger_kms.cc:310] Created kms service
I20260929 02:54:58.551810 8420 mini_ranger_kms.cc:326] Created kudu user
I20260929 02:54:58.591689 8420 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 02:54:58 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650496, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:54:58.979826 8420 mini_ranger_kms.cc:384] Added ranger policy
I20260929 02:54:58.980125 8420 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms:/tmp/dist-test-taskV6jRaD/build/asan/bin/postgresql.jar:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskV6jRaD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 02:54:58.980166 8420 mini_ranger_kms.cc:240] Using host: 127.8.57.20
I20260929 02:54:58.996371 8420 mini_ranger_kms.cc:276] Ranger KMS PID: 20667
I20260929 02:54:58.996474 8420 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.57.20:41339
Sep 29, 2026 2:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 2:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 2:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 2:55:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 02:55:01.043279 8420 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.008s user 0.062s sys 0.272s
I20260929 02:55:01.043537 8420 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 02:55:01.043713 8420 mini_ranger_kms.cc:401] 127.8.57.20:41339/kms/v1/keys
Sep 29 02:55:01 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650501, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.57.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 02:55:01 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650501, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.57.20@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
Sep 29 02:55:01 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650501, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.57.20@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
Sep 29 02:55:04 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:55:04.183408 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40669
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40669
--ranger_config_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:04.527977 20718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:04.528275 20718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:04.528400 20718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:04.535835 20718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:04.535991 20718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:04.536064 20718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:04.536139 20718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:04.536204 20718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:04.546075 20718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.57.20:41339/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40669
--ranger_config_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:04.547327 20718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:04.548682 20718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:04.561759 20723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:04.561733 20724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:04.562088 20726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:04.562470 20718 server_base.cc:1034] running on GCE node
I20260929 02:55:04.563241 20718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:04.564846 20718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:04.566046 20718 hybrid_clock.cc:648] HybridClock initialized: now 1790650504565981 us; error 71 us; skew 500 ppm
I20260929 02:55:04.566401 20718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:04 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:04.569602 20718 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:04.571115 20718 webserver.cc:460] Webserver started at http://127.8.57.62:36573/ using document root <none> and password file <none>
I20260929 02:55:04.571738 20718 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:04.571841 20718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:04.572147 20718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:55:04 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:04.620797 20718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "2ede5b385fd647a194aeb550b6502a03"
format_stamp: "Formatted at 2026-09-29 02:55:04 on dist-test-slave-1dkt"
server_key: "994430a6ab1efd9aed6d61b1ec4b54a6"
server_key_iv: "31461f09e96f0f107f4bad88d11b17d6"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:04.621789 20718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "2ede5b385fd647a194aeb550b6502a03"
format_stamp: "Formatted at 2026-09-29 02:55:04 on dist-test-slave-1dkt"
server_key: "994430a6ab1efd9aed6d61b1ec4b54a6"
server_key_iv: "31461f09e96f0f107f4bad88d11b17d6"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:04.627324 20718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:55:04.643141 20733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:04.644665 20718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:04.644842 20718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "2ede5b385fd647a194aeb550b6502a03"
format_stamp: "Formatted at 2026-09-29 02:55:04 on dist-test-slave-1dkt"
server_key: "994430a6ab1efd9aed6d61b1ec4b54a6"
server_key_iv: "31461f09e96f0f107f4bad88d11b17d6"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:04.645049 20718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:04.660199 20718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:04.664475 20718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:04.664762 20718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:04.682314 20718 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40669
I20260929 02:55:04.682344 20784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40669 every 8 connection(s)
I20260929 02:55:04.683614 20718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:04.687259 20785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:04.690241 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 20718
I20260929 02:55:04.690447 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:55:04.691205 8420 external_mini_cluster.cc:1428] Setting key b36e1a8c8134d7b0c7474b9bc6617e8c
I20260929 02:55:04.699740 20785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Bootstrap starting.
I20260929 02:55:04.703970 20785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:04.705433 20785 log.cc:824] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:04.711404 20785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: No bootstrap required, opened a new log
I20260929 02:55:04.720096 20785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:04.720450 20785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:04.720574 20785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ede5b385fd647a194aeb550b6502a03, State: Initialized, Role: FOLLOWER
I20260929 02:55:04.721153 20785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:04.721491 20785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:04.721663 20785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Sep 29 02:55:04 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:04.721832 20785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:04.724535 20785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:04.725070 20785 leader_election.cc:302] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ede5b385fd647a194aeb550b6502a03; no voters:
I20260929 02:55:04.725737 20785 leader_election.cc:288] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:55:04.725854 20790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:04.732125 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:04.693141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.8.57.62:40669)
0929 02:55:04.708579 (+ 15438us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:04.708599 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:04.708654 (+ 55us) server_negotiation.cc:407] Connection header received
0929 02:55:04.710828 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:04.710859 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:04.711333 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:04.711753 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:04.713324 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:04.715354 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:04.717160 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:04.718232 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:04.718416 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:04.723039 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:04.723049 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:04.723069 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:04.723119 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:04.726944 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:04.727752 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:04.727760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:04.727770 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:04.727968 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:04.728375 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:04.728380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:04.728383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:04.729032 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:04.730168 (+ 1136us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:04.730430 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 02:55:04.730810 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15162,"thread_start_us":14925,"threads_started":1}
I20260929 02:55:04.736509 20790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 LEADER]: Becoming Leader. State: Replica: 2ede5b385fd647a194aeb550b6502a03, State: Running, Role: LEADER
I20260929 02:55:04.737128 20790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:04.737716 20785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:04.739995 20791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ede5b385fd647a194aeb550b6502a03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } } }
I20260929 02:55:04.740260 20791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:04.740892 20792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ede5b385fd647a194aeb550b6502a03. Latest consensus state: current_term: 1 leader_uuid: "2ede5b385fd647a194aeb550b6502a03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } } }
I20260929 02:55:04.741226 20792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:04.742129 20796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:04.747002 20796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:04.757887 20796 catalog_manager.cc:1347] Generated new cluster ID: 1975be58136340e19f41a4b227a4bef2
I20260929 02:55:04.757992 20796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:04.801712 20796 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:55:04.803719 20796 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:04.813982 20796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Generated new TSK 0
I20260929 02:55:04.814942 20796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:55:04.817307 20785 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:55:05.672041 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40669
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:55:05 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 02:55:06.099589 20830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:06.100034 20830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:06.100247 20830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:06.111744 20830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:06.111961 20830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:06.112195 20830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:55:06.127163 20830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.57.20:41339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:06.129074 20830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:06.130997 20830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:06.155573 20841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:06.157945 20844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:06.161091 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:06.167889 20830 server_base.cc:1034] running on GCE node
I20260929 02:55:06.168877 20830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:06.170217 20830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:06.171519 20830 hybrid_clock.cc:648] HybridClock initialized: now 1790650506171456 us; error 55 us; skew 500 ppm
I20260929 02:55:06.171960 20830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:06.175976 20830 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:55:06.177814 20830 webserver.cc:460] Webserver started at http://127.8.57.1:37035/ using document root <none> and password file <none>
I20260929 02:55:06.178644 20830 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:06.178798 20830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:06.179136 20830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:06.204686 20830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e"
format_stamp: "Formatted at 2026-09-29 02:55:06 on dist-test-slave-1dkt"
server_key: "1519aaa67ead934e840f48a753351d46"
server_key_iv: "85781205e4a8fb65208c23bf59ba5b59"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:06.205506 20830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e"
format_stamp: "Formatted at 2026-09-29 02:55:06 on dist-test-slave-1dkt"
server_key: "1519aaa67ead934e840f48a753351d46"
server_key_iv: "85781205e4a8fb65208c23bf59ba5b59"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:06.213021 20830 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260929 02:55:06.245582 20850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:06.247737 20830 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.004s sys 0.016s
I20260929 02:55:06.248036 20830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e"
format_stamp: "Formatted at 2026-09-29 02:55:06 on dist-test-slave-1dkt"
server_key: "1519aaa67ead934e840f48a753351d46"
server_key_iv: "85781205e4a8fb65208c23bf59ba5b59"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:06.248413 20830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:06.276610 20830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:06.281981 20830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:06.282408 20830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:06.283769 20830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:06.285816 20830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:06.285964 20830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 02:55:06.286222 20830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:06.286346 20830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:06.363997 20830 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:36671
I20260929 02:55:06.364037 20962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:36671 every 8 connection(s)
I20260929 02:55:06.365389 20830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:55:06.374185 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 20830
I20260929 02:55:06.374403 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:55:06.375170 8420 external_mini_cluster.cc:1428] Setting key 3f33808c5487b964ae25628d791f376c
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:55:06.408558 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:06.370849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:41285 (local address 127.8.57.62:40669)
0929 02:55:06.386093 (+ 15244us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:06.386100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:06.386124 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:55:06.386187 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:06.386192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:06.386278 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:06.386399 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:06.388618 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:06.389707 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:06.392192 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:06.392902 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:06.393005 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:06.397568 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:06.397576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:06.397581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:06.397623 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:06.400674 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:06.401450 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:06.401457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:06.401460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:06.401611 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:06.406267 (+ 4656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:06.406282 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:06.406285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:06.406681 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:06.406845 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:06.407933 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0929 02:55:06.408176 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15096,"thread_start_us":15659,"threads_started":1}
I20260929 02:55:06.411132 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:06.370218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40669 (local address 127.8.57.1:41285)
0929 02:55:06.376507 (+ 6289us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:06.376560 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:06.378919 (+ 2359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:06.386788 (+ 7869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:06.386803 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:06.387489 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:06.388405 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:06.388434 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:06.389890 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:06.389896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:06.392021 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:06.392037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:06.393356 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:06.393364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:06.393644 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:06.394373 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:06.394410 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:06.397305 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:06.400816 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:06.400825 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:06.400839 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:06.401306 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:06.401762 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:06.401779 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:06.401783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:06.402026 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:06.406947 (+ 4921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:06.406965 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:06.407430 (+ 465us) client_negotiation.cc:769] Sending connection context
0929 02:55:06.408996 (+ 1566us) client_negotiation.cc:240] Negotiation successful
0929 02:55:06.409649 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5869,"thread_start_us":6702,"threads_started":1}
I20260929 02:55:06.412955 20963 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40669
I20260929 02:55:06.413295 20963 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:06.414168 20963 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:55:06.416783 20750 ts_manager.cc:194] Registered new tserver with Master: 6a7c1a2712554ff7a0b2f2c3758acd0e (127.8.57.1:36671)
I20260929 02:55:06.418726 20750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:41285
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:55:06.451504 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40669
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
W20260929 02:55:06.866340 20969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:06.866824 20969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:06.867018 20969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:06.878671 20969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:06.878895 20969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:06.879127 20969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:55:06.893911 20969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.57.20:41339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:06.895718 20969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:06.897437 20969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:06.936487 20977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:06.946652 20976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:06.952046 20969 server_base.cc:1034] running on GCE node
W20260929 02:55:06.952476 20980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:06.953379 20969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:06.962397 20969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:06.968989 20969 hybrid_clock.cc:648] HybridClock initialized: now 1790650506968837 us; error 148 us; skew 500 ppm
I20260929 02:55:06.969521 20969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:06.974088 20969 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:55:06.985406 20969 webserver.cc:460] Webserver started at http://127.8.57.2:35969/ using document root <none> and password file <none>
I20260929 02:55:06.986423 20969 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:06.986567 20969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:06.986945 20969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:55:06 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:07.020627 20969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "bc65eebe16334e2ab699954242e17fa1"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "b495a24ff8e25d93e6109b6248eb6d6c"
server_key_iv: "11b191245296c75351d20d2f19f8f332"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:07.021706 20969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "bc65eebe16334e2ab699954242e17fa1"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "b495a24ff8e25d93e6109b6248eb6d6c"
server_key_iv: "11b191245296c75351d20d2f19f8f332"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:07.028605 20969 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 02:55:07.048097 20993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:07.050064 20969 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:55:07.050316 20969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "bc65eebe16334e2ab699954242e17fa1"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "b495a24ff8e25d93e6109b6248eb6d6c"
server_key_iv: "11b191245296c75351d20d2f19f8f332"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:07.050659 20969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:07.085537 20969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:07.090528 20969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:07.090865 20969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:07.092039 20969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:07.093652 20969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:07.093741 20969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:07.093868 20969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:07.093924 20969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:07.126672 20969 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:43017
I20260929 02:55:07.126690 21105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:43017 every 8 connection(s)
I20260929 02:55:07.128198 20969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:55:07.129642 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 20969
I20260929 02:55:07.129786 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:55:07.130210 8420 external_mini_cluster.cc:1428] Setting key 9ebf8865d2c877b9cc3ab14862c14746
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:07.166261 21109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.134060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:43373 (local address 127.8.57.62:40669)
0929 02:55:07.144276 (+ 10216us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:07.144283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:07.144304 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:55:07.144368 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:07.144374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:07.144462 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:07.144574 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:07.146497 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.147622 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.149812 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.150421 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.150543 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.159488 (+ 8945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:07.159496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:07.159501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:07.159552 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:07.162903 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.163778 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.163783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.163787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.163961 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.164485 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.164491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.164494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.164788 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:07.164914 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:07.165637 (+ 723us) server_negotiation.cc:299] Negotiation successful
0929 02:55:07.165873 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10083,"thread_start_us":9975,"threads_started":1}
I20260929 02:55:07.167371 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.133517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40669 (local address 127.8.57.2:43373)
0929 02:55:07.138449 (+ 4932us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:07.138501 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:07.140708 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:07.144902 (+ 4194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:07.144915 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:07.145510 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:07.146301 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:07.146326 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.147781 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.147786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:07.149627 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:07.149642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.150643 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.150649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:07.150933 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.154630 (+ 3697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:07.154667 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:07.159182 (+ 4515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:07.163064 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:07.163073 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:07.163092 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:07.163641 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:07.164066 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:07.164071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:07.164075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:07.164338 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:07.164925 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:07.164934 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:07.165262 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 02:55:07.165510 (+ 248us) client_negotiation.cc:240] Negotiation successful
0929 02:55:07.166050 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4622,"thread_start_us":4395,"threads_started":1}
I20260929 02:55:07.168848 21106 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40669
I20260929 02:55:07.169157 21106 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:07.169941 21106 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
I20260929 02:55:07.171682 20750 ts_manager.cc:194] Registered new tserver with Master: bc65eebe16334e2ab699954242e17fa1 (127.8.57.2:43017)
I20260929 02:55:07.172786 20750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:43373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:55:07.185388 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40669
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:07.422322 20963 heartbeater.cc:503] Master 127.8.57.62:40669 was elected leader, sending a full tablet report...
W20260929 02:55:07.496255 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:07.496500 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:07.496587 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:07.503463 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:07.503559 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:07.503682 21112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:55:07.513008 21112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.57.20:41339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:07.514156 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:07.515313 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:07.526870 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:07.526907 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:07.526883 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:07.527899 21112 server_base.cc:1034] running on GCE node
I20260929 02:55:07.528431 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:07.529260 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:07.530452 21112 hybrid_clock.cc:648] HybridClock initialized: now 1790650507530407 us; error 35 us; skew 500 ppm
I20260929 02:55:07.530719 21112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:07.533566 21112 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:55:07.534811 21112 webserver.cc:460] Webserver started at http://127.8.57.3:42889/ using document root <none> and password file <none>
I20260929 02:55:07.535408 21112 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:07.535509 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:07.535763 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:07.551270 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "74b96f2add8249288690a30ae042b324"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "5636757376821856526685306fd1e548"
server_key_iv: "9eadc1adce2f6306f79fa57d2df4a1b0"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:07.552261 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "74b96f2add8249288690a30ae042b324"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "5636757376821856526685306fd1e548"
server_key_iv: "9eadc1adce2f6306f79fa57d2df4a1b0"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:07.557809 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:55:07.572494 21127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:07.573906 21112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:07.574054 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "74b96f2add8249288690a30ae042b324"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "5636757376821856526685306fd1e548"
server_key_iv: "9eadc1adce2f6306f79fa57d2df4a1b0"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:07.574244 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:07.595706 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:07.599279 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:07.599525 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:07.600396 21112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:07.601631 21112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:07.601698 21112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:07.601811 21112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:07.601857 21112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:07.630226 21112 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:43951
I20260929 02:55:07.630270 21239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:43951 every 8 connection(s)
I20260929 02:55:07.631539 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:55:07.637295 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21112
I20260929 02:55:07.637475 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:55:07.637945 8420 external_mini_cluster.cc:1428] Setting key 7c1c5f595ca8327c784caf1a45fbcf62
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:07.649722 21109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.633919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:57669 (local address 127.8.57.62:40669)
0929 02:55:07.634092 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:07.634097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:07.635195 (+ 1098us) server_negotiation.cc:407] Connection header received
0929 02:55:07.637010 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:07.637016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:07.637121 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:07.637206 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:07.638692 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.639487 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.641303 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.641700 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.641805 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.644859 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:07.644865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:07.644868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:07.644899 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:07.647084 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.647704 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.647708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.647710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.647838 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.648258 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.648262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.648264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.648540 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:07.648660 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:07.649282 (+ 622us) server_negotiation.cc:299] Negotiation successful
0929 02:55:07.649491 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:55:07.650554 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.634361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40669 (local address 127.8.57.3:57669)
0929 02:55:07.635051 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:07.635091 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:07.636757 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:07.637456 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:07.637464 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:07.637901 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:07.638475 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:07.638492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.639644 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.639647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:07.641162 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:07.641180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.641901 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.641908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:07.642124 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.642696 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:07.642716 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:07.644653 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:07.647203 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:07.647210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:07.647222 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:07.647601 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:07.647969 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:07.647972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:07.647974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:07.648152 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:07.648640 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:07.648648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:07.648937 (+ 289us) client_negotiation.cc:769] Sending connection context
0929 02:55:07.649140 (+ 203us) client_negotiation.cc:240] Negotiation successful
0929 02:55:07.649544 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":278,"threads_started":1}
I20260929 02:55:07.651834 21240 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40669
I20260929 02:55:07.652158 21240 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:07.652917 21240 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
I20260929 02:55:07.654166 20750 ts_manager.cc:194] Registered new tserver with Master: 74b96f2add8249288690a30ae042b324 (127.8.57.3:43951)
I20260929 02:55:07.654990 20750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:57669
I20260929 02:55:07.664875 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:55:07.666524 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 20830
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:55:07.682996 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.667422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39142 (local address 127.8.57.1:36671)
0929 02:55:07.668001 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:07.668007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:07.668030 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:55:07.668080 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:07.668086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:07.668280 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:07.668414 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:07.669762 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.671170 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.673086 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.673612 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.673725 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.677631 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:07.677645 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:07.677661 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:07.677700 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:07.680275 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.680966 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.680973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.680975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.681106 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.681460 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.681464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.681466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.681843 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:07.682137 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:07.682419 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:55:07.682655 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":279,"threads_started":1}
I20260929 02:55:07.684136 20958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:07.765107 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 20830
I20260929 02:55:07.776841 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 20969
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:55:07.791458 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.777642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.8.57.2:43017)
0929 02:55:07.778202 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:07.778210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:07.778235 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:55:07.778304 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:07.778310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:07.778536 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:07.778675 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:07.779754 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.780891 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.782163 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.782688 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.782775 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.786717 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:07.786727 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:07.786741 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:07.786789 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:07.789151 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.789728 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.789734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.789736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.789861 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.790121 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.790124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.790126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.790423 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:07.790631 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:07.790917 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 02:55:07.791107 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":324,"threads_started":1}
I20260929 02:55:07.792675 21101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:07.872319 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 20969
I20260929 02:55:07.884013 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21112
Sep 29 02:55:07 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:55:07.899199 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:07.884857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.8.57.3:43951)
0929 02:55:07.885419 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:07.885425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:07.885454 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:55:07.885515 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:07.885520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:07.885734 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:07.885860 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:07.887118 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.888188 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.889450 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:07.890013 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:07.890111 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:07.894292 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:07.894304 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:07.894325 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:07.894398 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:07.896972 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.897584 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.897591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.897593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.897712 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:07.897979 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:07.897981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:07.897983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:07.898283 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:07.898521 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:07.898724 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 02:55:07.898916 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":269,"threads_started":1}
I20260929 02:55:07.900410 21235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:07.980424 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21112
I20260929 02:55:07.992385 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 20718
I20260929 02:55:07.993552 20780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:08.099464 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 20718
I20260929 02:55:08.120287 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40669
--webserver_interface=127.8.57.62
--webserver_port=36573
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40669
--ranger_config_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:08.423560 21251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:08.423841 21251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:08.423983 21251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:08.431061 21251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:08.431167 21251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:08.431241 21251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:08.431316 21251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:08.431381 21251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:08.441282 21251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40669
--ranger_config_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=36573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:08.442503 21251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:08.443753 21251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:08.454977 21259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:08.454983 21257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:08.455394 21256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:08.455847 21251 server_base.cc:1034] running on GCE node
I20260929 02:55:08.456573 21251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:08.457886 21251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:08.459080 21251 hybrid_clock.cc:648] HybridClock initialized: now 1790650508459032 us; error 44 us; skew 500 ppm
I20260929 02:55:08.459388 21251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:08 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:08.462217 21251 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:08.463532 21251 webserver.cc:460] Webserver started at http://127.8.57.62:36573/ using document root <none> and password file <none>
I20260929 02:55:08.464231 21251 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:08.464334 21251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:55:08 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:08.508740 21251 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 02:55:08.511781 21267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:08.513228 21251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:55:08.513393 21251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "2ede5b385fd647a194aeb550b6502a03"
format_stamp: "Formatted at 2026-09-29 02:55:04 on dist-test-slave-1dkt"
server_key: "994430a6ab1efd9aed6d61b1ec4b54a6"
server_key_iv: "31461f09e96f0f107f4bad88d11b17d6"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:08.514317 21251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:08.525652 21251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:08.529083 21251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:08.529321 21251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:08.545809 21251 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40669
I20260929 02:55:08.545827 21318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40669 every 8 connection(s)
I20260929 02:55:08.547173 21251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:08.549217 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21251
I20260929 02:55:08.550181 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:36671
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:40669
--webserver_port=37035
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:08.555743 21319 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:55:08.559388 21319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Bootstrap starting.
I20260929 02:55:08.577661 21319 log.cc:824] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:08.586171 21319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:55:08.586633 21319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Bootstrap complete.
I20260929 02:55:08.594954 21319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:08.595760 21319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ede5b385fd647a194aeb550b6502a03, State: Initialized, Role: FOLLOWER
I20260929 02:55:08.596379 21319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:08.596588 21319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:08.596689 21319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:55:08.596834 21319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:55:08.600410 21319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:08.600912 21319 leader_election.cc:302] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ede5b385fd647a194aeb550b6502a03; no voters:
I20260929 02:55:08.601497 21319 leader_election.cc:288] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:55:08.601645 21323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:55:08.602667 21323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [term 2 LEADER]: Becoming Leader. State: Replica: 2ede5b385fd647a194aeb550b6502a03, State: Running, Role: LEADER
I20260929 02:55:08.603233 21323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } }
I20260929 02:55:08.603500 21319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:08.605509 21325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ede5b385fd647a194aeb550b6502a03. Latest consensus state: current_term: 2 leader_uuid: "2ede5b385fd647a194aeb550b6502a03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } } }
I20260929 02:55:08.605860 21325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:08.606362 21324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2ede5b385fd647a194aeb550b6502a03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede5b385fd647a194aeb550b6502a03" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40669 } } }
I20260929 02:55:08.606631 21324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:08.611115 21330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:08.616937 21330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:08.618486 21330 catalog_manager.cc:1259] Loaded cluster ID: 1975be58136340e19f41a4b227a4bef2
I20260929 02:55:08.618678 21330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:08.622740 21330 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:08.624264 21330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Loaded TSK: 0
I20260929 02:55:08.625887 21330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:55:08.628454 21319 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260929 02:55:08.970688 21321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:08.971179 21321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:08.971369 21321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:08.982703 21321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:08.982851 21321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:08.983093 21321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:55:08.998413 21321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:36671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=37035
--tserver_master_addrs=127.8.57.62:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:09.000196 21321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:09.001967 21321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:09.057147 21357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:09.059662 21356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:09.063211 21321 server_base.cc:1034] running on GCE node
W20260929 02:55:09.063508 21359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:09.064303 21321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:09.065609 21321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:09.069753 21321 hybrid_clock.cc:648] HybridClock initialized: now 1790650509068124 us; error 1621 us; skew 500 ppm
I20260929 02:55:09.070186 21321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:09.074317 21321 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:55:09.086503 21321 webserver.cc:460] Webserver started at http://127.8.57.1:37035/ using document root <none> and password file <none>
I20260929 02:55:09.087563 21321 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:09.087738 21321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:09.154045 21321 fs_manager.cc:709] Time spent opening directory manager: real 0.043s user 0.004s sys 0.024s
I20260929 02:55:09.165601 21368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:09.167668 21321 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.010s
I20260929 02:55:09.167969 21321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e"
format_stamp: "Formatted at 2026-09-29 02:55:06 on dist-test-slave-1dkt"
server_key: "1519aaa67ead934e840f48a753351d46"
server_key_iv: "85781205e4a8fb65208c23bf59ba5b59"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:09.169775 21321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:09.202826 21321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:09.208077 21321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:09.208427 21321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:09.213423 21321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:09.215150 21321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:09.215261 21321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:09.215436 21321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:09.215529 21321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:09.311475 21321 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:36671
I20260929 02:55:09.311566 21480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:36671 every 8 connection(s)
I20260929 02:55:09.313674 21321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:55:09.318961 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21321
I20260929 02:55:09.319736 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:43017
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:40669
--webserver_port=35969
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:09.376807 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:09.325279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:60129 (local address 127.8.57.62:40669)
0929 02:55:09.348353 (+ 23074us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:09.348371 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:09.348436 (+ 65us) server_negotiation.cc:407] Connection header received
0929 02:55:09.350770 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:09.350791 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:09.351312 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:09.351748 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:09.354174 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.356104 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.358491 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.359500 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.359667 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:09.364048 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:09.364059 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:09.364078 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:09.364125 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:09.367908 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:09.369086 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:09.369095 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:09.369130 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:09.369329 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:09.371270 (+ 1941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:09.371276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:09.371279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:09.371987 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:09.372169 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:09.374642 (+ 2473us) server_negotiation.cc:299] Negotiation successful
0929 02:55:09.375048 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22785,"thread_start_us":22571,"threads_started":1}
I20260929 02:55:09.377485 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:09.326090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40669 (local address 127.8.57.1:60129)
0929 02:55:09.345766 (+ 19676us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:09.345820 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:09.350038 (+ 4218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:09.352184 (+ 2146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:09.352195 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:09.352825 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:09.353655 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:09.353680 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.356306 (+ 2626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.356312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:09.358306 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:09.358333 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.360045 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.360053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:09.360310 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:09.361065 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:09.361093 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:09.363751 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:09.368074 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:09.368082 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:09.368098 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:09.368856 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:09.370857 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:09.370863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:09.370866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:09.371122 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:09.372241 (+ 1119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:09.372249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:09.372632 (+ 383us) client_negotiation.cc:769] Sending connection context
0929 02:55:09.372936 (+ 304us) client_negotiation.cc:240] Negotiation successful
0929 02:55:09.373498 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19337,"thread_start_us":19138,"threads_started":1}
I20260929 02:55:09.381844 21481 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40669
I20260929 02:55:09.382180 21481 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:09.383028 21481 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
W20260929 02:55:09.386310 21481 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:40669 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 02:55:09.386538 21481 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:09.386838 21481 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
I20260929 02:55:09.388239 21481 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:09.388588 21481 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
W20260929 02:55:09.389633 21481 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:55:09.563144 21492 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260929 02:55:09.741570 21484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:09.741850 21484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:09.741986 21484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:09.749975 21484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:09.750157 21484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:09.750368 21484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:55:09.765650 21484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:43017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=35969
--tserver_master_addrs=127.8.57.62:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:09.767364 21484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:09.769133 21484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:09.795248 21501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:09.801170 21503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:09.803081 21499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:09.804226 21484 server_base.cc:1034] running on GCE node
I20260929 02:55:09.805168 21484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:09.806934 21484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:09.808151 21484 hybrid_clock.cc:648] HybridClock initialized: now 1790650509808045 us; error 95 us; skew 500 ppm
I20260929 02:55:09.808547 21484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:09.812587 21484 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:55:09.820190 21484 webserver.cc:460] Webserver started at http://127.8.57.2:35969/ using document root <none> and password file <none>
I20260929 02:55:09.820871 21484 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:09.820996 21484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:09.853331 21484 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.009s sys 0.000s
I20260929 02:55:09.856211 21512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:09.857568 21484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:55:09.857729 21484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "bc65eebe16334e2ab699954242e17fa1"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "b495a24ff8e25d93e6109b6248eb6d6c"
server_key_iv: "11b191245296c75351d20d2f19f8f332"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:09.858680 21484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:09.884101 21484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:09.891711 21484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:09.892009 21484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:09.893076 21484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:09.904763 21484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:09.904927 21484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:09.905076 21484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:09.905184 21484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:09.942426 21484 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:43017
I20260929 02:55:09.942471 21624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:43017 every 8 connection(s)
I20260929 02:55:09.944056 21484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:55:09.946832 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21484
I20260929 02:55:09.947410 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.57.20:41339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:43951
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:40669
--webserver_port=42889
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:55:09 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:09.977059 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:09.952158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54315 (local address 127.8.57.62:40669)
0929 02:55:09.952631 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:09.952638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:09.952660 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:55:09.952724 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:09.952729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:09.952818 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:09.952936 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:09.960026 (+ 7090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.961186 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.964382 (+ 3196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.965075 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.965177 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:09.969648 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:09.969655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:09.969660 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:09.969701 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:09.973319 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:09.974249 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:09.974255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:09.974259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:09.974444 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:09.974980 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:09.974985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:09.974989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:09.975343 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:09.975474 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:09.976465 (+ 991us) server_negotiation.cc:299] Negotiation successful
0929 02:55:09.976689 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":229,"threads_started":1}
I20260929 02:55:09.979615 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:09.947567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40669 (local address 127.8.57.2:54315)
0929 02:55:09.948421 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:09.948469 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:09.950626 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:09.955051 (+ 4425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:09.955064 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:09.955670 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:09.956508 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:09.956534 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.961421 (+ 4887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.961427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:09.964186 (+ 2759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:09.964212 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:09.965553 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:09.965560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:09.965882 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:09.966649 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:09.966684 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:09.969373 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:09.973497 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:09.973507 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:09.973525 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:09.974108 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:09.974596 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:09.974601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:09.974605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:09.974846 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:09.975565 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:09.975573 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:09.976003 (+ 430us) client_negotiation.cc:769] Sending connection context
0929 02:55:09.977637 (+ 1634us) client_negotiation.cc:240] Negotiation successful
0929 02:55:09.978182 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":390,"threads_started":1}
I20260929 02:55:09.981473 21625 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40669
I20260929 02:55:09.981803 21625 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:09.982692 21625 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
I20260929 02:55:09.984735 21284 ts_manager.cc:194] Registered new tserver with Master: bc65eebe16334e2ab699954242e17fa1 (127.8.57.2:43017)
I20260929 02:55:09.986893 21284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:54315
Sep 29 02:55:10 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:10.389767 21481 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:10.390183 21481 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
I20260929 02:55:10.391422 21284 ts_manager.cc:194] Registered new tserver with Master: 6a7c1a2712554ff7a0b2f2c3758acd0e (127.8.57.1:36671)
I20260929 02:55:10.392555 21284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:60129
W20260929 02:55:10.416680 21630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:10.417145 21630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:10.417392 21630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:10.428588 21630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:10.428781 21630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:10.428999 21630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:55:10.443982 21630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.57.20:41339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:43951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=42889
--tserver_master_addrs=127.8.57.62:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:10.445534 21630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:10.447086 21630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:10.462167 21635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:10.463843 21636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:10.470978 21638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:10.471539 21630 server_base.cc:1034] running on GCE node
I20260929 02:55:10.472263 21630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:10.473388 21630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:10.474669 21630 hybrid_clock.cc:648] HybridClock initialized: now 1790650510474599 us; error 62 us; skew 500 ppm
I20260929 02:55:10.475031 21630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:10 dist-test-slave-1dkt krb5kdc[20274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650510, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:10.478943 21630 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:55:10.482196 21630 webserver.cc:460] Webserver started at http://127.8.57.3:42889/ using document root <none> and password file <none>
I20260929 02:55:10.483064 21630 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:10.483201 21630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:55:10 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650510, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:10.509133 21630 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:55:10.512665 21646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:10.514396 21630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:10.514573 21630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "74b96f2add8249288690a30ae042b324"
format_stamp: "Formatted at 2026-09-29 02:55:07 on dist-test-slave-1dkt"
server_key: "5636757376821856526685306fd1e548"
server_key_iv: "9eadc1adce2f6306f79fa57d2df4a1b0"
server_key_version: "kuduclusterkey@0"
I20260929 02:55:10.516039 21630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:10.534910 21630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:10.539781 21630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:10.540141 21630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:10.541388 21630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:10.543103 21630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:10.543216 21630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:10.543354 21630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:10.543454 21630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:55:10 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for HTTP/127.8.57.20@KRBTEST.COM
I20260929 02:55:10.864156 21630 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:43951
I20260929 02:55:10.864207 21766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:43951 every 8 connection(s)
I20260929 02:55:10.865870 21630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:55:10.867063 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21630
Sep 29 02:55:10 dist-test-slave-1dkt krb5kdc[20274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650510, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:10.909835 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.868171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.8.57.62:40669)
0929 02:55:10.890406 (+ 22235us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:10.890414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:10.896054 (+ 5640us) server_negotiation.cc:407] Connection header received
0929 02:55:10.896249 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:10.896253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:10.896309 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:10.896403 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:10.897975 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.898787 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.900138 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.900590 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.900688 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.903258 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:10.903265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:10.903268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:10.903297 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:10.907080 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:10.907858 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:10.907864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:10.907868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:10.908044 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:10.908431 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:10.908434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:10.908436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:10.908786 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:10.909058 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:10.909313 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 02:55:10.909474 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":22072,"spinlock_wait_cycles":1408,"thread_start_us":27196,"threads_started":1}
I20260929 02:55:10.910261 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.869742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:38937 (local address 127.8.57.62:40669)
0929 02:55:10.895670 (+ 25928us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:10.895676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:10.895695 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:55:10.895761 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:10.895766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:10.895843 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:10.895977 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:10.897597 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.898332 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.899862 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.900314 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.900423 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.903644 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:10.903651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:10.903657 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:10.903709 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:10.907071 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:10.907858 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:10.907864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:10.907868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:10.908033 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:10.908516 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:10.908520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:10.908523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:10.908814 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:10.908967 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:10.909690 (+ 723us) server_negotiation.cc:299] Negotiation successful
0929 02:55:10.909872 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":25790,"thread_start_us":20684,"threads_started":1}
I20260929 02:55:10.911293 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.869099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40669 (local address 127.8.57.3:38937)
0929 02:55:10.891128 (+ 22029us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:10.891199 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:10.893954 (+ 2755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:10.896245 (+ 2291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:10.896257 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:10.896814 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:10.897440 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.897458 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.898478 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.898482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.899731 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.899749 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.900538 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.900545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.900757 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.901371 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:10.901419 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:10.903412 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:10.907199 (+ 3787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:10.907207 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:10.907223 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:10.907748 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:10.908139 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:10.908143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:10.908146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:10.908423 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:10.908917 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:10.908924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:10.909295 (+ 371us) client_negotiation.cc:769] Sending connection context
0929 02:55:10.909610 (+ 315us) client_negotiation.cc:240] Negotiation successful
0929 02:55:10.910104 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21650,"thread_start_us":25208,"threads_started":1}
I20260929 02:55:10.912933 21767 heartbeater.cc:347] Connected to a master server at 127.8.57.62:40669
I20260929 02:55:10.913221 21767 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:10.913841 21767 heartbeater.cc:511] Master 127.8.57.62:40669 requested a full tablet report, sending...
I20260929 02:55:10.915061 21284 ts_manager.cc:194] Registered new tserver with Master: 74b96f2add8249288690a30ae042b324 (127.8.57.3:43951)
I20260929 02:55:10.915812 21284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:38937
I20260929 02:55:10.924608 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:55:10.938302 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.927194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.8.57.62:40669)
0929 02:55:10.927404 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:10.927410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:10.927659 (+ 249us) server_negotiation.cc:407] Connection header received
0929 02:55:10.927774 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:10.927778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:10.927831 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:10.927955 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:10.929222 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.929908 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.931098 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.931481 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.931564 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.933895 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:10.933902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:10.933906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:10.933942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:10.936331 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:10.936941 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:10.936945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:10.936947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:10.937058 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:10.937301 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:10.937304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:10.937306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:10.937549 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:10.937727 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:10.937942 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 02:55:10.938095 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:55:10.942821 21284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:55:10.951829 21284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:55:10.988982 21782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.964572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:43951 (local address 127.0.0.1:41134)
0929 02:55:10.971762 (+ 7190us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:10.971804 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:10.972007 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:10.981303 (+ 9296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:10.981311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:10.981342 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:10.981748 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.981761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.983686 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.983691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.985922 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.985939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.988090 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.988097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.988279 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.988328 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 02:55:10.988455 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 02:55:10.988635 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6521,"thread_start_us":6373,"threads_started":1}
I20260929 02:55:10.989305 21784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.964932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43017 (local address 127.0.0.1:59966)
0929 02:55:10.976994 (+ 12062us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:10.977015 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:10.977140 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:10.982379 (+ 5239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:10.982384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:10.982406 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:10.982818 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.982830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.984618 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.984624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.986687 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.986701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.988504 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.988511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.988674 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.988706 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:55:10.988801 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 02:55:10.988936 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11918,"thread_start_us":11790,"threads_started":1}
I20260929 02:55:10.989470 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.964826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.8.57.3:43951)
0929 02:55:10.980704 (+ 15878us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:10.980714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:10.980749 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:55:10.980843 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:10.980850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:10.981090 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:10.981259 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:10.981974 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.983532 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.986109 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.987533 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.987703 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.987944 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:10.988718 (+ 774us) server_negotiation.cc:299] Negotiation successful
0929 02:55:10.988872 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15685,"thread_start_us":18960,"threads_started":1}
I20260929 02:55:10.990145 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.965278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.8.57.2:43017)
0929 02:55:10.981923 (+ 16645us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:10.981930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:10.981950 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:55:10.982022 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:10.982028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:10.982218 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:10.982373 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:10.983030 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.984481 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.986881 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.988292 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.988415 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.988619 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:10.989685 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0929 02:55:10.989838 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16490,"thread_start_us":16359,"threads_started":1}
I20260929 02:55:10.990276 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.977084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:36671 (local address 127.0.0.1:40850)
0929 02:55:10.982957 (+ 5873us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:10.982972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:10.983071 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:10.984045 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:10.984050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:10.984078 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:10.984435 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.984446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.985734 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.985738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.987496 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:10.987507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.989241 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.989247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:10.989436 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.989466 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:55:10.989856 (+ 390us) client_negotiation.cc:240] Negotiation successful
0929 02:55:10.989972 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5757,"thread_start_us":5601,"threads_started":1}
I20260929 02:55:10.990921 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:10.972185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.8.57.1:36671)
0929 02:55:10.981113 (+ 8928us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:10.981123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:10.983180 (+ 2057us) server_negotiation.cc:407] Connection header received
0929 02:55:10.983276 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:10.983282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:10.983494 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:10.983659 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:10.984666 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.985610 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.987625 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:10.988803 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:10.988900 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:10.989101 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:10.990465 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0929 02:55:10.990623 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8743,"thread_start_us":8558,"threads_started":1}
I20260929 02:55:10.991547 21625 heartbeater.cc:503] Master 127.8.57.62:40669 was elected leader, sending a full tablet report...
I20260929 02:55:10.993234 21695 tablet_service.cc:1467] Processing CreateTablet for tablet ac33ba46d0ad45af9d883d64235fe887 (DEFAULT_TABLE table=test-table [id=ace9d11be5004afeb086930a0f987bb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:10.993861 21560 tablet_service.cc:1467] Processing CreateTablet for tablet ac33ba46d0ad45af9d883d64235fe887 (DEFAULT_TABLE table=test-table [id=ace9d11be5004afeb086930a0f987bb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:10.994187 21416 tablet_service.cc:1467] Processing CreateTablet for tablet ac33ba46d0ad45af9d883d64235fe887 (DEFAULT_TABLE table=test-table [id=ace9d11be5004afeb086930a0f987bb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:10.994863 21695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac33ba46d0ad45af9d883d64235fe887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:10.995122 21560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac33ba46d0ad45af9d883d64235fe887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:10.995306 21416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac33ba46d0ad45af9d883d64235fe887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:11.006677 21788 tablet_bootstrap.cc:492] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Bootstrap starting.
I20260929 02:55:11.009266 21789 tablet_bootstrap.cc:492] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Bootstrap starting.
I20260929 02:55:11.010051 21788 tablet_bootstrap.cc:654] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:11.011361 21788 log.cc:824] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:11.012825 21789 tablet_bootstrap.cc:654] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:11.014089 21789 log.cc:824] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:11.015232 21790 tablet_bootstrap.cc:492] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Bootstrap starting.
I20260929 02:55:11.017151 21789 tablet_bootstrap.cc:492] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: No bootstrap required, opened a new log
I20260929 02:55:11.017432 21789 ts_tablet_manager.cc:1397] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260929 02:55:11.017964 21788 tablet_bootstrap.cc:492] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: No bootstrap required, opened a new log
I20260929 02:55:11.017999 21790 tablet_bootstrap.cc:654] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:11.018213 21788 ts_tablet_manager.cc:1397] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260929 02:55:11.018959 21790 log.cc:824] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:11.021690 21790 tablet_bootstrap.cc:492] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: No bootstrap required, opened a new log
I20260929 02:55:11.021948 21790 ts_tablet_manager.cc:1397] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:55:11.027669 21789 raft_consensus.cc:348] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.028144 21789 raft_consensus.cc:374] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:11.028298 21789 raft_consensus.cc:729] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc65eebe16334e2ab699954242e17fa1, State: Initialized, Role: FOLLOWER
I20260929 02:55:11.028228 21788 raft_consensus.cc:348] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.028692 21788 raft_consensus.cc:374] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:11.028798 21788 raft_consensus.cc:729] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a7c1a2712554ff7a0b2f2c3758acd0e, State: Initialized, Role: FOLLOWER
I20260929 02:55:11.028689 21790 raft_consensus.cc:348] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.029033 21790 raft_consensus.cc:374] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:11.028924 21789 consensus_queue.cc:260] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.029166 21790 raft_consensus.cc:729] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b96f2add8249288690a30ae042b324, State: Initialized, Role: FOLLOWER
I20260929 02:55:11.029358 21788 consensus_queue.cc:260] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.029728 21790 consensus_queue.cc:260] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.030712 21767 heartbeater.cc:503] Master 127.8.57.62:40669 was elected leader, sending a full tablet report...
I20260929 02:55:11.031651 21790 ts_tablet_manager.cc:1428] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260929 02:55:11.031801 21789 ts_tablet_manager.cc:1428] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 02:55:11.032667 21481 heartbeater.cc:503] Master 127.8.57.62:40669 was elected leader, sending a full tablet report...
I20260929 02:55:11.033541 21788 ts_tablet_manager.cc:1428] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
I20260929 02:55:11.073081 21795 raft_consensus.cc:482] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:55:11.073321 21795 raft_consensus.cc:504] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.074822 21795 leader_election.cc:288] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a7c1a2712554ff7a0b2f2c3758acd0e (127.8.57.1:36671), bc65eebe16334e2ab699954242e17fa1 (127.8.57.2:43017)
I20260929 02:55:11.080504 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.075236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:58949 (local address 127.8.57.1:36671)
0929 02:55:11.075463 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:11.075470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:11.075690 (+ 220us) server_negotiation.cc:407] Connection header received
0929 02:55:11.075908 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:11.075914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:11.076032 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:11.076144 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:11.076490 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.077194 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.078820 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.079696 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.079805 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.079938 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:11.080164 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 02:55:11.080280 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:55:11.080514 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.075248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:36671 (local address 127.8.57.3:58949)
0929 02:55:11.075601 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:11.075620 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:11.075752 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:11.076162 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:11.076167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:11.076195 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:11.076356 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:11.076364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.077320 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.077324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:11.078667 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:11.078678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.079828 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.079833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:11.080024 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.080048 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:55:11.080125 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:55:11.080245 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260929 02:55:11.081485 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.075286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:37511 (local address 127.8.57.2:43017)
0929 02:55:11.075487 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:11.075493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:11.076386 (+ 893us) server_negotiation.cc:407] Connection header received
0929 02:55:11.076524 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:11.076527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:11.076591 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:11.076660 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:11.077139 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.077824 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.079720 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.080514 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.080633 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.080729 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:11.081130 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 02:55:11.081251 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:55:11.081508 21797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.075601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43017 (local address 127.8.57.3:37511)
0929 02:55:11.076292 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:11.076310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:11.076422 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:11.076683 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:11.076687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:11.076708 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:11.077009 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:11.077017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.077960 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.077966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:11.079580 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:11.079592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.080810 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.080816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:11.080988 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.081020 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:55:11.081104 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:55:11.081213 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":376,"threads_started":1}
I20260929 02:55:11.081418 21436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33ba46d0ad45af9d883d64235fe887" candidate_uuid: "74b96f2add8249288690a30ae042b324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" is_pre_election: true
I20260929 02:55:11.081909 21436 raft_consensus.cc:2454] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74b96f2add8249288690a30ae042b324 in term 0.
I20260929 02:55:11.082357 21580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33ba46d0ad45af9d883d64235fe887" candidate_uuid: "74b96f2add8249288690a30ae042b324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc65eebe16334e2ab699954242e17fa1" is_pre_election: true
I20260929 02:55:11.082505 21649 leader_election.cc:302] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a7c1a2712554ff7a0b2f2c3758acd0e, 74b96f2add8249288690a30ae042b324; no voters:
I20260929 02:55:11.082784 21580 raft_consensus.cc:2454] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74b96f2add8249288690a30ae042b324 in term 0.
I20260929 02:55:11.082851 21795 raft_consensus.cc:2781] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:55:11.082971 21795 raft_consensus.cc:482] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:55:11.083070 21795 raft_consensus.cc:3037] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:11.085240 21795 raft_consensus.cc:504] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.085738 21795 leader_election.cc:288] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [CANDIDATE]: Term 1 election: Requested vote from peers 6a7c1a2712554ff7a0b2f2c3758acd0e (127.8.57.1:36671), bc65eebe16334e2ab699954242e17fa1 (127.8.57.2:43017)
I20260929 02:55:11.086102 21436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33ba46d0ad45af9d883d64235fe887" candidate_uuid: "74b96f2add8249288690a30ae042b324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e"
I20260929 02:55:11.086189 21580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac33ba46d0ad45af9d883d64235fe887" candidate_uuid: "74b96f2add8249288690a30ae042b324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc65eebe16334e2ab699954242e17fa1"
I20260929 02:55:11.086259 21436 raft_consensus.cc:3037] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:11.086369 21580 raft_consensus.cc:3037] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:11.088393 21436 raft_consensus.cc:2454] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74b96f2add8249288690a30ae042b324 in term 1.
I20260929 02:55:11.088514 21580 raft_consensus.cc:2454] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74b96f2add8249288690a30ae042b324 in term 1.
I20260929 02:55:11.088729 21649 leader_election.cc:302] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a7c1a2712554ff7a0b2f2c3758acd0e, 74b96f2add8249288690a30ae042b324; no voters:
I20260929 02:55:11.088996 21795 raft_consensus.cc:2781] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:11.089394 21795 raft_consensus.cc:686] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 1 LEADER]: Becoming Leader. State: Replica: 74b96f2add8249288690a30ae042b324, State: Running, Role: LEADER
I20260929 02:55:11.089802 21795 consensus_queue.cc:237] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } }
I20260929 02:55:11.093822 21282 catalog_manager.cc:5495] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74b96f2add8249288690a30ae042b324 (127.8.57.3). New cstate: current_term: 1 leader_uuid: "74b96f2add8249288690a30ae042b324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b96f2add8249288690a30ae042b324" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 43951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 } health_report { overall_health: UNKNOWN } } }
W20260929 02:55:11.096832 21485 tablet.cc:2369] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:11.113889 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.108775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.8.57.3:43951)
0929 02:55:11.108954 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:11.108960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:11.109045 (+ 85us) server_negotiation.cc:407] Connection header received
0929 02:55:11.109251 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:11.109255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:11.109324 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:11.109454 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:55:11.109816 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.110511 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.111909 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.112339 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.112450 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.112926 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:55:11.113386 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:55:11.113539 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:11.113590 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 02:55:11.113664 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 02:55:11.117442 21768 tablet.cc:2369] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:11.121307 21436 raft_consensus.cc:1261] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 1 FOLLOWER]: Refusing update from remote peer 74b96f2add8249288690a30ae042b324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:11.121316 21580 raft_consensus.cc:1261] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 1 FOLLOWER]: Refusing update from remote peer 74b96f2add8249288690a30ae042b324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:11.122152 21798 consensus_queue.cc:1035] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc65eebe16334e2ab699954242e17fa1" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:11.122469 21795 consensus_queue.cc:1035] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a7c1a2712554ff7a0b2f2c3758acd0e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 36671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:11.137336 21802 mvcc.cc:205] Tried to move back new op lower bound from 7334504493544046592 to 7334504493427441664. Current Snapshot: MvccSnapshot[applied={T|T < 7334504493544046592}]
I20260929 02:55:11.138661 21801 mvcc.cc:205] Tried to move back new op lower bound from 7334504493544046592 to 7334504493427441664. Current Snapshot: MvccSnapshot[applied={T|T < 7334504493544046592}]
I20260929 02:55:11.150403 21282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:11.150633 21282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:11.153635 21282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ede5b385fd647a194aeb550b6502a03: Sending DeleteTablet for 3 replicas of tablet ac33ba46d0ad45af9d883d64235fe887
I20260929 02:55:11.154654 21560 tablet_service.cc:1514] Processing DeleteTablet for tablet ac33ba46d0ad45af9d883d64235fe887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:11 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:59966
I20260929 02:55:11.154645 21416 tablet_service.cc:1514] Processing DeleteTablet for tablet ac33ba46d0ad45af9d883d64235fe887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:11 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:40850
I20260929 02:55:11.154726 21695 tablet_service.cc:1514] Processing DeleteTablet for tablet ac33ba46d0ad45af9d883d64235fe887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:11 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:41134
I20260929 02:55:11.155560 21811 tablet_replica.cc:330] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: stopping tablet replica
I20260929 02:55:11.156081 21812 tablet_replica.cc:330] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: stopping tablet replica
I20260929 02:55:11.156508 21811 raft_consensus.cc:2229] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:11.156944 21812 raft_consensus.cc:2229] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:11.157415 21812 raft_consensus.cc:2258] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:11.157855 21811 raft_consensus.cc:2258] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:11.159688 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21321
I20260929 02:55:11.160135 21811 ts_tablet_manager.cc:1905] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:11.166752 21811 ts_tablet_manager.cc:1918] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:11.167354 21813 tablet_replica.cc:330] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: stopping tablet replica
I20260929 02:55:11.167762 21813 raft_consensus.cc:2229] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:55:11.168435 21813 raft_consensus.cc:2258] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:11.169749 21812 ts_tablet_manager.cc:1905] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:11.170212 21813 ts_tablet_manager.cc:1905] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:11.174826 21811 log.cc:1196] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/ac33ba46d0ad45af9d883d64235fe887
I20260929 02:55:11.175408 21811 ts_tablet_manager.cc:1939] T ac33ba46d0ad45af9d883d64235fe887 P 6a7c1a2712554ff7a0b2f2c3758acd0e: Deleting consensus metadata
I20260929 02:55:11.177253 21270 catalog_manager.cc:4841] TS 6a7c1a2712554ff7a0b2f2c3758acd0e (127.8.57.1:36671): tablet ac33ba46d0ad45af9d883d64235fe887 (table test-table [id=ace9d11be5004afeb086930a0f987bb7]) successfully deleted
I20260929 02:55:11.179440 21813 ts_tablet_manager.cc:1918] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:11.179606 21813 log.cc:1196] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/ac33ba46d0ad45af9d883d64235fe887
I20260929 02:55:11.180120 21813 ts_tablet_manager.cc:1939] T ac33ba46d0ad45af9d883d64235fe887 P 74b96f2add8249288690a30ae042b324: Deleting consensus metadata
I20260929 02:55:11.180768 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.162457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.8.57.1:36671)
0929 02:55:11.162625 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:11.162630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:11.162649 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:55:11.162701 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:11.162706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:11.162784 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:11.162870 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:11.168844 (+ 5974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.169548 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.171269 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.171689 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.171790 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.174748 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:11.174758 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:11.174770 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:11.174797 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:11.177820 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:11.178540 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:11.178546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:11.178549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:11.178661 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:11.179050 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:11.179055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:11.179058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:11.179388 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:11.179663 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:11.180114 (+ 451us) server_negotiation.cc:299] Negotiation successful
0929 02:55:11.180305 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:55:11.181300 21476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:11.181627 21270 catalog_manager.cc:4841] TS 74b96f2add8249288690a30ae042b324 (127.8.57.3:43951): tablet ac33ba46d0ad45af9d883d64235fe887 (table test-table [id=ace9d11be5004afeb086930a0f987bb7]) successfully deleted
I20260929 02:55:11.186195 21812 ts_tablet_manager.cc:1918] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:11.186345 21812 log.cc:1196] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/ac33ba46d0ad45af9d883d64235fe887
I20260929 02:55:11.186849 21812 ts_tablet_manager.cc:1939] T ac33ba46d0ad45af9d883d64235fe887 P bc65eebe16334e2ab699954242e17fa1: Deleting consensus metadata
I20260929 02:55:11.188412 21268 catalog_manager.cc:4841] TS bc65eebe16334e2ab699954242e17fa1 (127.8.57.2:43017): tablet ac33ba46d0ad45af9d883d64235fe887 (table test-table [id=ace9d11be5004afeb086930a0f987bb7]) successfully deleted
I20260929 02:55:11.269867 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21321
I20260929 02:55:11.282936 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21484
I20260929 02:55:11.295684 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.283664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.8.57.2:43017)
0929 02:55:11.283859 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:11.283864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:11.283906 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:55:11.284051 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:11.284057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:11.284125 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:11.284227 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:11.285435 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.286147 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.287347 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.287754 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.287858 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.290285 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:11.290297 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:11.290311 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:11.290340 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:11.293125 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:11.293899 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:11.293905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:11.293908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:11.294036 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:11.294420 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:11.294424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:11.294427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:11.294748 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:11.294969 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:11.295264 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 02:55:11.295410 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:55:11.296375 21620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:11.447770 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21484
I20260929 02:55:11.465958 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21630
I20260929 02:55:11.481508 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:11.466761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.8.57.3:43951)
0929 02:55:11.466980 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:11.466986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:11.467029 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:55:11.467169 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:11.467175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:11.467259 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:11.467395 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:11.468629 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.469790 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.471060 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:11.471696 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:11.471833 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:11.474533 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:11.474550 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:11.474568 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:11.474614 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:11.478479 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:11.479193 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:11.479203 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:11.479206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:11.479407 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:11.479824 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:11.479830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:11.479833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:11.480302 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:11.480630 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:11.480852 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 02:55:11.481093 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 02:55:11.482336 21762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:11.607429 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21630
I20260929 02:55:11.622056 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21251
I20260929 02:55:11.623200 21314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:11.733763 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21251
I20260929 02:55:11.750798 8420 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 02:55:12.084492 8420 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 02:55:12.084672 8420 mini_ranger.cc:66] Stopping Ranger...
I20260929 02:55:12.419706 8420 mini_ranger.cc:68] Stopped Ranger
2026-09-29 02:55:12.419 UTC [20304] LOG: received smart shutdown request
2026-09-29 02:55:12.423 UTC [20304] LOG: background worker "logical replication launcher" (PID 20313) exited with exit code 1
2026-09-29 02:55:12.423 UTC [20308] LOG: shutting down
2026-09-29 02:55:12.572 UTC [20304] LOG: database system is shut down
2026-09-29T02:55:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49425 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:55:12 dist-test-slave-1dkt krb5kdc[21826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:55:12 dist-test-slave-1dkt krb5kdc[21826](info): set up 2 sockets
Sep 29 02:55:12 dist-test-slave-1dkt krb5kdc[21826](info): commencing operation
krb5kdc: starting...
W20260929 02:55:13.692512 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:55:13 dist-test-slave-1dkt krb5kdc[21826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:55:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:55:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:55:13.884828 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37569
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37569,127.8.57.61:38809,127.8.57.60:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:14.193627 21839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:14.193903 21839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:14.194038 21839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:14.201157 21839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:14.201257 21839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:14.201328 21839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:14.201406 21839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:14.201462 21839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:14.210961 21839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37569,127.8.57.61:38809,127.8.57.60:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:14.212241 21839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:14.213492 21839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:14.224751 21844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:14.224740 21847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:14.224707 21845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:14.225381 21839 server_base.cc:1034] running on GCE node
I20260929 02:55:14.226078 21839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:14.227340 21839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:14.228525 21839 hybrid_clock.cc:648] HybridClock initialized: now 1790650514228487 us; error 36 us; skew 500 ppm
I20260929 02:55:14.228824 21839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:14 dist-test-slave-1dkt krb5kdc[21826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:14.231678 21839 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:14.233038 21839 webserver.cc:460] Webserver started at http://127.8.57.62:35261/ using document root <none> and password file <none>
I20260929 02:55:14.233647 21839 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:14.233749 21839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:14.234035 21839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:14.236853 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "7afa77707e34496092088221889988f2"
format_stamp: "Formatted at 2026-09-29 02:55:14 on dist-test-slave-1dkt"
server_key: "9847ed5956fb1c583941a704153480d5"
server_key_iv: "cca77605da07e342b3ee3e0abf23e160"
server_key_version: "encryptionkey@0"
I20260929 02:55:14.237602 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "7afa77707e34496092088221889988f2"
format_stamp: "Formatted at 2026-09-29 02:55:14 on dist-test-slave-1dkt"
server_key: "9847ed5956fb1c583941a704153480d5"
server_key_iv: "cca77605da07e342b3ee3e0abf23e160"
server_key_version: "encryptionkey@0"
I20260929 02:55:14.242268 21839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 02:55:14.245415 21854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:14.246649 21839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:55:14.246793 21839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "7afa77707e34496092088221889988f2"
format_stamp: "Formatted at 2026-09-29 02:55:14 on dist-test-slave-1dkt"
server_key: "9847ed5956fb1c583941a704153480d5"
server_key_iv: "cca77605da07e342b3ee3e0abf23e160"
server_key_version: "encryptionkey@0"
I20260929 02:55:14.246987 21839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:14.277360 21839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:14.281325 21839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:14.281587 21839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:14.298503 21839 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:37569
I20260929 02:55:14.298542 21905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:37569 every 8 connection(s)
I20260929 02:55:14.299749 21839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:14.302623 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21839
I20260929 02:55:14.302837 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:55:14.303608 8420 external_mini_cluster.cc:1428] Setting key b26dc7737cd13672136b8d2e3f1eaaff
I20260929 02:55:14.303685 21906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:14.312137 21906 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.57.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.61@KRBTEST.COM" created.
Sep 29 02:55:14 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:14.335378 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.315107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52610 (local address 127.8.57.62:37569)
0929 02:55:14.316916 (+ 1809us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:14.316922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:14.316974 (+ 52us) server_negotiation.cc:407] Connection header received
0929 02:55:14.318711 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:14.318718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:14.318936 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:14.319096 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:14.320544 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.321750 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.323724 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.324302 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.324462 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:14.328195 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:14.328206 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:14.328220 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:14.328258 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:14.331414 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:14.332115 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:14.332122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:14.332125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:14.332300 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:14.332644 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:14.332648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:14.332651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:14.333245 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:14.333447 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:14.333891 (+ 444us) server_negotiation.cc:299] Negotiation successful
0929 02:55:14.334221 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":1633,"spinlock_wait_cycles":69376,"thread_start_us":1409,"threads_started":1}
I20260929 02:55:14.335410 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.315107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37569 (local address 127.0.0.1:52610)
0929 02:55:14.316046 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.316093 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:14.318353 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:14.319156 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:14.319168 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:14.319895 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:14.320340 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:14.320364 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.321911 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.321917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:14.323589 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:14.323605 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.324529 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.324536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:14.324688 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:14.325275 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:14.325297 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:14.327964 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:14.331551 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:14.331560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:14.331569 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:14.331987 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:14.332399 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:14.332402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:14.332405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:14.332553 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:14.333349 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:14.333354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:14.333535 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 02:55:14.333742 (+ 207us) client_negotiation.cc:240] Negotiation successful
0929 02:55:14.334338 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":129,"thread_start_us":406,"threads_started":1}
I20260929 02:55:14.338275 21906 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:55:14.339294 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.338887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35822)
0929 02:55:14.339065 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.339174 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260929 02:55:14.339566 21855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
I20260929 02:55:14.341091 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.340701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35838)
0929 02:55:14.340849 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.340944 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 02:55:14.341298 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:38809: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab.
Entry for principal kudu/127.8.57.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab.
I20260929 02:55:14.353071 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:38809
--webserver_interface=127.8.57.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab
--principal=kudu/127.8.57.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37569,127.8.57.61:38809,127.8.57.60:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:14.390594 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } attempt: 1
I20260929 02:55:14.391753 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.391327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35854)
0929 02:55:14.391523 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.391625 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 02:55:14.392735 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.392411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35866)
0929 02:55:14.392537 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.392604 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 02:55:14.392920 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:38809: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
I20260929 02:55:14.461145 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } attempt: 2
I20260929 02:55:14.462280 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.461834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35878)
0929 02:55:14.462055 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.462145 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 02:55:14.463177 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.462865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35890)
0929 02:55:14.462972 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.463051 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 02:55:14.463366 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:38809: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
I20260929 02:55:14.554590 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } attempt: 3
I20260929 02:55:14.555698 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.555277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35896)
0929 02:55:14.555478 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.555566 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 02:55:14.556655 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.556343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35912)
0929 02:55:14.556468 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.556532 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:55:14.556845 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:38809: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
W20260929 02:55:14.656831 21913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:14.657105 21913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:14.657228 21913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:14.664213 21913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:14.664306 21913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:14.664379 21913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:14.664448 21913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:14.664503 21913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:14.673792 21913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37569,127.8.57.61:38809,127.8.57.60:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab
--principal=kudu/127.8.57.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.57.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:14.674995 21913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:14.676258 21913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:14.687074 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:14.687101 21921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:14.687150 21919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:14.688005 21913 server_base.cc:1034] running on GCE node
I20260929 02:55:14.688680 21913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:14.689971 21913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:14.691200 21913 hybrid_clock.cc:648] HybridClock initialized: now 1790650514691152 us; error 33 us; skew 500 ppm
I20260929 02:55:14.691540 21913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:14 dist-test-slave-1dkt krb5kdc[21826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:14.694756 21913 init.cc:377] Logged in from keytab as kudu/127.8.57.61@KRBTEST.COM (short username kudu)
I20260929 02:55:14.696600 21913 webserver.cc:460] Webserver started at http://127.8.57.61:35109/ using document root <none> and password file <none>
I20260929 02:55:14.697273 21913 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:14.697376 21913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:14.697661 21913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:14.700047 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } attempt: 4
I20260929 02:55:14.700631 21913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data/instance:
uuid: "43b387c041154046bf5168811dd9ddac"
format_stamp: "Formatted at 2026-09-29 02:55:14 on dist-test-slave-1dkt"
server_key: "5e90645ac19b1299e865088d7a374232"
server_key_iv: "82b349eaa467c4c2e0f8568c715c1bd1"
server_key_version: "encryptionkey@0"
I20260929 02:55:14.701118 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.700607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35928)
0929 02:55:14.700899 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.700981 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260929 02:55:14.701498 21913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal/instance:
uuid: "43b387c041154046bf5168811dd9ddac"
format_stamp: "Formatted at 2026-09-29 02:55:14 on dist-test-slave-1dkt"
server_key: "5e90645ac19b1299e865088d7a374232"
server_key_iv: "82b349eaa467c4c2e0f8568c715c1bd1"
server_key_version: "encryptionkey@0"
I20260929 02:55:14.702059 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.701724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35930)
0929 02:55:14.701853 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.701943 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 02:55:14.702214 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:38809: Network error: Client connection negotiation failed: client connection to 127.8.57.61:38809: connect: Connection refused (error 111)
I20260929 02:55:14.706319 21913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:55:14.709648 21928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:14.710943 21913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:55:14.711098 21913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal
uuid: "43b387c041154046bf5168811dd9ddac"
format_stamp: "Formatted at 2026-09-29 02:55:14 on dist-test-slave-1dkt"
server_key: "5e90645ac19b1299e865088d7a374232"
server_key_iv: "82b349eaa467c4c2e0f8568c715c1bd1"
server_key_version: "encryptionkey@0"
I20260929 02:55:14.711269 21913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:14.730952 21913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:14.735136 21913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:14.735401 21913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:14.752478 21913 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.61:38809
I20260929 02:55:14.752497 21979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.61:38809 every 8 connection(s)
I20260929 02:55:14.753895 21913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
I20260929 02:55:14.757769 21980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:14.761200 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21913
I20260929 02:55:14.761404 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-1/wal/instance
I20260929 02:55:14.761843 8420 external_mini_cluster.cc:1428] Setting key 74ba4e70ebb138b3c24f22a7501d6818
I20260929 02:55:14.764807 21980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:55:14 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.60@KRBTEST.COM" created.
I20260929 02:55:14.785995 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.767550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.8.57.62:37569)
0929 02:55:14.767774 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:14.767781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:14.769005 (+ 1224us) server_negotiation.cc:407] Connection header received
0929 02:55:14.770922 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:14.770927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:14.771011 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:14.771119 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:14.772707 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.773691 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.775357 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.776027 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.776152 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:14.779848 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:14.779854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:14.779857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:14.779922 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:14.782695 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:14.783704 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:14.783710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:14.783713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:14.783829 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:14.784301 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:14.784305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:14.784307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:14.784607 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:14.784719 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:14.785585 (+ 866us) server_negotiation.cc:299] Negotiation successful
0929 02:55:14.785759 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:55:14.787423 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.768010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37569 (local address 127.0.0.1:52616)
0929 02:55:14.768819 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.768867 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:14.770647 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:14.771329 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:14.771337 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:14.771867 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:14.772517 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:14.772536 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.773880 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.773885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:14.775200 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:14.775214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.776292 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.776300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:14.776547 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:14.777293 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:14.777317 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:14.779613 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:14.782870 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:14.782887 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:14.782907 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:14.783558 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:14.783960 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:14.783965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:14.783968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:14.784197 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:14.784732 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:14.784740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:14.785192 (+ 452us) client_negotiation.cc:769] Sending connection context
0929 02:55:14.785501 (+ 309us) client_negotiation.cc:240] Negotiation successful
0929 02:55:14.786025 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":339,"threads_started":1}
I20260929 02:55:14.789149 21980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:55:14 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:55:14.803105 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.789901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35934)
0929 02:55:14.790084 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.790100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:14.790185 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:14.791227 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:14.791232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:14.791641 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:14.791772 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:14.791780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.793112 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.793116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:14.793926 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:14.793937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.794698 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.794703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:14.794899 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:14.795478 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:14.795501 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:14.797730 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:14.800679 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:14.800684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:14.800689 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:14.801041 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:14.801370 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:14.801375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:14.801378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:14.801519 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:14.802204 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:14.802208 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:14.802399 (+ 191us) client_negotiation.cc:769] Sending connection context
0929 02:55:14.802523 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 02:55:14.802724 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 02:55:14.803349 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.790189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.8.57.61:38809)
0929 02:55:14.790796 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:14.790802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:14.790827 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:55:14.790869 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:14.790874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:14.791079 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:14.791212 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:14.791941 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.792974 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.794049 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:14.794569 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:14.794676 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:14.797915 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:14.797927 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:14.797943 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:14.797978 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:14.800565 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:14.801157 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:14.801163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:14.801166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:14.801274 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:14.801660 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:14.801664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:14.801666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:14.802058 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:14.802224 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:14.802674 (+ 450us) server_negotiation.cc:299] Negotiation successful
0929 02:55:14.802857 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":347,"threads_started":1}
I20260929 02:55:14.805922 21980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:55:14.807152 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.806622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44656)
0929 02:55:14.806780 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.806988 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 02:55:14.807427 21931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
I20260929 02:55:14.808876 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.808586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44666)
0929 02:55:14.808697 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.808774 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 02:55:14.809129 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab.
Entry for principal kudu/127.8.57.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab.
I20260929 02:55:14.811156 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:43497
--webserver_interface=127.8.57.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab
--principal=kudu/127.8.57.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37569,127.8.57.61:38809,127.8.57.60:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:14.858479 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 1
I20260929 02:55:14.859746 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.859274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44672)
0929 02:55:14.859486 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.859590 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 02:55:14.860827 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.860503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44678)
0929 02:55:14.860647 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.860718 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 02:55:14.861039 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
I20260929 02:55:14.929266 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 2
I20260929 02:55:14.930424 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.929966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44680)
0929 02:55:14.930162 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.930257 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 02:55:14.931530 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:14.931146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44686)
0929 02:55:14.931308 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:14.931403 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 02:55:14.931747 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
I20260929 02:55:15.001480 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } attempt: 5
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:55:15.015743 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.002437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.8.57.61:38809)
0929 02:55:15.002595 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.002600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.002638 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:55:15.002695 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.002699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.002773 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.002845 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.003838 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.004574 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.006225 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.006663 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.006756 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.010371 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.010376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.010380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.010414 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.012959 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.013805 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.013809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.013811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.013948 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.014337 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.014340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.014342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.014688 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.014801 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.015266 (+ 465us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.015452 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:55:15.015962 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.002199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35942)
0929 02:55:15.002420 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.002443 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:15.002574 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:15.002905 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:15.002910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:15.003434 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:15.003675 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.003697 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.004717 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.004723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.006086 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.006111 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.006817 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.006839 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.007037 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.007643 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:15.007665 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:15.010169 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:15.013111 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.013117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.013121 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.013692 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.014052 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.014057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.014061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.014239 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.014805 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:15.014838 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:15.015025 (+ 187us) client_negotiation.cc:769] Sending connection context
0929 02:55:15.015187 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 02:55:15.015517 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 02:55:15.017032 21906 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:55:15.018131 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.017669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44696)
0929 02:55:15.017852 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.017953 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 02:55:15.019063 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.018711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44700)
0929 02:55:15.018860 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.018943 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 02:55:15.019254 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
I20260929 02:55:15.022928 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 3
I20260929 02:55:15.023815 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.023455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44712)
0929 02:55:15.023604 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.023677 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 02:55:15.024784 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.024401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44722)
0929 02:55:15.024604 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.024673 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260929 02:55:15.024959 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
I20260929 02:55:15.070492 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 1
I20260929 02:55:15.071655 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.071188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44728)
0929 02:55:15.071393 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.071486 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 02:55:15.072745 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.072397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44740)
0929 02:55:15.072562 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.072633 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260929 02:55:15.072971 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
W20260929 02:55:15.135115 21987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:15.135516 21987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:15.135663 21987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 02:55:15.141192 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 2
I20260929 02:55:15.142437 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.141896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44754)
0929 02:55:15.142162 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.142259 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260929 02:55:15.143458 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.143103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44760)
0929 02:55:15.143264 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.143342 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:55:15.143666 21906 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
W20260929 02:55:15.145222 21987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:15.145344 21987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:15.145429 21987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:15.145505 21987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:15.145565 21987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:15.156415 21987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37569,127.8.57.61:38809,127.8.57.60:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab
--principal=kudu/127.8.57.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.57.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:15.157927 21987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:15.159287 21987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:55:15.168152 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 4
I20260929 02:55:15.169302 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.168784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44766)
0929 02:55:15.169022 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.169120 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 02:55:15.170562 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.170053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44780)
0929 02:55:15.170264 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.170347 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260929 02:55:15.170810 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:43497: Network error: Client connection negotiation failed: client connection to 127.8.57.60:43497: connect: Connection refused (error 111)
W20260929 02:55:15.171401 21993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:15.171371 21995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:15.171339 21992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:15.172262 21987 server_base.cc:1034] running on GCE node
I20260929 02:55:15.173035 21987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:15.174428 21987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:15.175654 21987 hybrid_clock.cc:648] HybridClock initialized: now 1790650515175594 us; error 43 us; skew 500 ppm
I20260929 02:55:15.176048 21987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650515, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:15.179100 21987 init.cc:377] Logged in from keytab as kudu/127.8.57.60@KRBTEST.COM (short username kudu)
I20260929 02:55:15.180552 21987 webserver.cc:460] Webserver started at http://127.8.57.60:36329/ using document root <none> and password file <none>
I20260929 02:55:15.181222 21987 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:15.181342 21987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:15.181648 21987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:15.184616 21987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data/instance:
uuid: "80c33542f22c408d9754bff6c9b7b89b"
format_stamp: "Formatted at 2026-09-29 02:55:15 on dist-test-slave-1dkt"
server_key: "ef4fb97a9b02b4273c9ab1b00906f4f2"
server_key_iv: "9a66606db4842853dacbf2c9970486a6"
server_key_version: "encryptionkey@0"
I20260929 02:55:15.185379 21987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal/instance:
uuid: "80c33542f22c408d9754bff6c9b7b89b"
format_stamp: "Formatted at 2026-09-29 02:55:15 on dist-test-slave-1dkt"
server_key: "ef4fb97a9b02b4273c9ab1b00906f4f2"
server_key_iv: "9a66606db4842853dacbf2c9970486a6"
server_key_version: "encryptionkey@0"
I20260929 02:55:15.190404 21987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:55:15.193884 22002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:15.195341 21987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:55:15.195515 21987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal
uuid: "80c33542f22c408d9754bff6c9b7b89b"
format_stamp: "Formatted at 2026-09-29 02:55:15 on dist-test-slave-1dkt"
server_key: "ef4fb97a9b02b4273c9ab1b00906f4f2"
server_key_iv: "9a66606db4842853dacbf2c9970486a6"
server_key_version: "encryptionkey@0"
I20260929 02:55:15.195703 21987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:15.216773 21987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:15.220669 21987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:15.220945 21987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:15.237670 21987 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.60:43497
I20260929 02:55:15.237674 22053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.60:43497 every 8 connection(s)
I20260929 02:55:15.239035 21987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
I20260929 02:55:15.241151 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 21987
I20260929 02:55:15.241348 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650321430198-8420-0/minicluster-data/master-2/wal/instance
I20260929 02:55:15.241886 8420 external_mini_cluster.cc:1428] Setting key c5659350b1289e0d16b09b9a232cded8
I20260929 02:55:15.243325 22054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:15.249853 21906 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 3
I20260929 02:55:15.252415 22054 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650515, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:15.264994 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.243713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.8.57.60:43497)
0929 02:55:15.244488 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.244500 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.244540 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:55:15.246074 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.246097 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.246485 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.246817 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.248621 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.250133 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.251805 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.252855 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.253058 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.257335 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.257344 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.257353 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.257385 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.260725 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.261502 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.261510 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.261518 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.261725 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.262165 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.262169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.262171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.262683 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.262978 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.263421 (+ 443us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.263856 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":373,"threads_started":1}
I20260929 02:55:15.266150 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.250425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44800)
0929 02:55:15.250607 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.250626 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:15.250716 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:15.251542 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:15.251547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:15.252097 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:15.252381 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.252396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.253888 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.253894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.255104 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.255117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.256109 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.256116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.256295 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.256864 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:15.256886 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:15.259622 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:15.263432 (+ 3810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.263437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.263441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.263933 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.264359 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.264364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.264367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.264514 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.265146 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:15.265151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:15.265350 (+ 199us) client_negotiation.cc:769] Sending connection context
0929 02:55:15.265475 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 02:55:15.265750 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:55:15.267037 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.250557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.8.57.60:43497)
0929 02:55:15.251224 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.251230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.251251 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:55:15.251292 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.251297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.251364 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.251468 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.252580 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.253703 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.255265 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.255930 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.256628 (+ 698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.259823 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.259829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.259834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.259867 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.263283 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.264092 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.264097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.264101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.264241 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.264627 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.264632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.264636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.265004 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.266497 (+ 1493us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.266602 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.266735 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":375,"threads_started":1}
I20260929 02:55:15.271688 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.255330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.8.57.62:37569)
0929 02:55:15.255500 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.255505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.256157 (+ 652us) server_negotiation.cc:407] Connection header received
0929 02:55:15.256397 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.256402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.256488 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.256554 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.258053 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.258747 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.260350 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.260979 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.261096 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.265272 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.265278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.265286 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.265320 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.268494 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.269570 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.269576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.269580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.269785 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.270198 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.270203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.270206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.270594 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.270731 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.271148 (+ 417us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.271333 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:55:15.271688 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.255323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:37569 (local address 127.0.0.1:52626)
0929 02:55:15.256040 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.256087 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:15.256269 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:15.256781 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:15.256788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:15.257530 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:15.257885 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.257897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.258937 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.258941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.260197 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.260210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.261167 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.261175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.261361 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.262287 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:15.262308 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:15.265096 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:15.268618 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.268658 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.268667 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.269454 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.269925 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.269930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.269934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.270082 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.270708 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:15.270716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:15.270868 (+ 152us) client_negotiation.cc:769] Sending connection context
0929 02:55:15.271006 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 02:55:15.271291 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"spinlock_wait_cycles":16640,"thread_start_us":318,"threads_started":1}
I20260929 02:55:15.273090 21906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: Bootstrap starting.
I20260929 02:55:15.273204 22054 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:55:15.276810 21906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:15.277763 21906 log.cc:824] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:15.280871 21906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: No bootstrap required, opened a new log
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650515, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:55:15.285521 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.269559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.8.57.62:37569)
0929 02:55:15.270108 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.270113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.270130 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:55:15.270177 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.270182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.270255 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.270357 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.271856 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.273046 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.274703 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.275151 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.275255 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.279789 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.279794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.279797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.279825 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.282835 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.283515 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.283521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.283525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.283660 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.284095 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.284100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.284103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.284393 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.284655 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.284910 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.285127 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":455,"thread_start_us":276,"threads_started":1}
I20260929 02:55:15.288002 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.273914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:38809 (local address 127.0.0.1:35954)
0929 02:55:15.274156 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.274173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:15.274274 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:15.274613 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:15.274618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:15.275036 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:15.275197 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.275208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.276564 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.276569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.277904 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.277918 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.278762 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.278768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.278941 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.279506 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:15.279527 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:15.281920 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:15.285538 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.285544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.285548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.286011 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.286372 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.286376 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.286380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.286554 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.287158 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:15.287161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:15.287268 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 02:55:15.287372 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 02:55:15.287631 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260929 02:55:15.288094 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.273946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35954 (local address 127.8.57.61:38809)
0929 02:55:15.274110 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.274116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.274236 (+ 120us) server_negotiation.cc:407] Connection header received
0929 02:55:15.274387 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.274393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.274476 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.274576 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.275383 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.276402 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.278062 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.278618 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.278727 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.282118 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.282124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.282129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.282167 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.285421 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.286129 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.286134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.286138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.286271 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.286676 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.286681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.286684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.286989 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.287086 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.287512 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.287710 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:55:15.288930 22054 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:55:15.291741 21906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.292171 21906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:15.292282 21906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7afa77707e34496092088221889988f2, State: Initialized, Role: FOLLOWER
I20260929 02:55:15.293133 21906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650515, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
I20260929 02:55:15.301450 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.301862 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:55:15.302821 21906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:15.306627 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.287407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.8.57.61:38809)
0929 02:55:15.288287 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.288293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.288308 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:55:15.288346 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.288350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.288413 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.289987 (+ 1574us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.290116 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.291219 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.293067 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.293672 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.294328 (+ 656us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.299318 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.299324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.299328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.299364 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.302932 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.303778 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.303784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.303788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.303971 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.304373 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.304378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.304382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.304699 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.304972 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.306067 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.306274 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":13381,"threads_started":1}
I20260929 02:55:15.311483 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:55:15.313279 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.289683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44806)
0929 02:55:15.290175 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.290191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:15.290304 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:15.290710 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:15.290714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:15.291132 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:15.291303 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.291314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.292606 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.292611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.293865 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.293878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.295137 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.295142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.295304 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.295852 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:15.295893 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:15.304762 (+ 8869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:15.309537 (+ 4775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.309542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.309547 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.310043 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.310404 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.310408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.310412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.310540 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.312397 (+ 1857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:15.312399 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:15.312516 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 02:55:15.312631 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 02:55:15.312906 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418}
I20260929 02:55:15.313616 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.289943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.8.57.60:43497)
0929 02:55:15.290084 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.290090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.290239 (+ 149us) server_negotiation.cc:407] Connection header received
0929 02:55:15.290404 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.290408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.290489 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.290550 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.291419 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.292466 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.293987 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.294853 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.294941 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.304989 (+ 10048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.304996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.305001 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.305039 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.309371 (+ 4332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.310155 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.310161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.310164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.310288 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.310635 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.310640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.310644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.312239 (+ 1595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.312340 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.312738 (+ 398us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.313056 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":50}
I20260929 02:55:15.318022 22054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: Bootstrap starting.
I20260929 02:55:15.321022 22054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:15.322074 22054 log.cc:824] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:15.325765 22054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: No bootstrap required, opened a new log
I20260929 02:55:15.337312 22054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.337764 22054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:15.337886 22054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c33542f22c408d9754bff6c9b7b89b, State: Initialized, Role: FOLLOWER
I20260929 02:55:15.338676 22054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.345497 22081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.345791 22081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:55:15.357576 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.324106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.8.57.61:38809)
0929 02:55:15.324279 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.324286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.334249 (+ 9963us) server_negotiation.cc:407] Connection header received
0929 02:55:15.334686 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.334692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.334777 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.334846 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.337130 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.338157 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.341010 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.341633 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.341723 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.347581 (+ 5858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.347588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.347592 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.347644 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.351277 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.355224 (+ 3947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.355229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.355233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.355395 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.355822 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.355827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.355830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.356227 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.356457 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.357011 (+ 554us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.357230 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:55:15.360653 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.330221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.8.57.60:43497)
0929 02:55:15.330384 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.330390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.340826 (+ 10436us) server_negotiation.cc:407] Connection header received
0929 02:55:15.340979 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.340986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.341092 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.341191 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.343482 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.344629 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.349045 (+ 4416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.349736 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.349859 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.354487 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.354493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.354497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.354534 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.357719 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.358518 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.358523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.358526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.358662 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.359051 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.359056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.359059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.359376 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.359613 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.360117 (+ 504us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.360326 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:55:15.361428 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.324027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.8.57.62:37569)
0929 02:55:15.324245 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.324251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.330962 (+ 6711us) server_negotiation.cc:407] Connection header received
0929 02:55:15.331426 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.331432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.331531 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.331604 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.336343 (+ 4739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.337340 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.341301 (+ 3961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.341922 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.342010 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.350760 (+ 8750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.350767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.350772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.350813 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.354490 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.359111 (+ 4621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.359116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.359120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.359268 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.359720 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.359724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.359728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.360093 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.360334 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.360842 (+ 508us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.361069 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:55:15.362231 22054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:15.351609 21839 master_runner.cc:186] Error getting master registration for 127.8.57.61:38809: OK, instance_id { permanent_uuid: "43b387c041154046bf5168811dd9ddac" instance_seqno: 1790650514747316 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:55:15.365875 21839 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 02:55:15.366758 22083 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:55:15.366988 22083 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260929 02:55:15.378444 22095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:55:15.378592 22095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:55:15.376269 21987 master_runner.cc:186] Error getting master registration for 127.8.57.61:38809: OK, instance_id { permanent_uuid: "43b387c041154046bf5168811dd9ddac" instance_seqno: 1790650514747316 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:55:15.388197 21987 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:55:15.470058 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } attempt: 5
W20260929 02:55:15.472992 22008 tablet.cc:2369] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:55:15 dist-test-slave-1dkt krb5kdc[21826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650514, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
I20260929 02:55:15.487066 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.470701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:43497 (local address 127.0.0.1:44830)
0929 02:55:15.471020 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:15.471040 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:15.471161 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:15.471515 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:15.471520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:15.472069 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:15.472275 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.472287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.473946 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.473952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.475408 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:15.475424 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.476722 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.476728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:15.476927 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.477592 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:15.477616 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:15.480404 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:15.484365 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.484371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.484376 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.484882 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.485268 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:15.485274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:15.485277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:15.485456 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:15.486047 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:15.486052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:15.486220 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 02:55:15.486373 (+ 153us) client_negotiation.cc:240] Negotiation successful
0929 02:55:15.486660 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260929 02:55:15.487075 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:15.470802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.8.57.60:43497)
0929 02:55:15.471000 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:15.471006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:15.471115 (+ 109us) server_negotiation.cc:407] Connection header received
0929 02:55:15.471282 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:15.471288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:15.471363 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:15.471482 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:15.472435 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.473711 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.475590 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:15.476382 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:15.476508 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:15.480598 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:15.480604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:15.480610 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:15.480646 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:15.484237 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.485010 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.485016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.485019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.485159 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:15.485566 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:15.485571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:15.485575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:15.485924 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:15.486077 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:15.486483 (+ 406us) server_negotiation.cc:299] Negotiation successful
0929 02:55:15.486706 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:55:15.493062 21980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: Bootstrap starting.
I20260929 02:55:15.496229 22063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:55:15.496511 22063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.497781 21980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:15.498046 22063 leader_election.cc:288] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43b387c041154046bf5168811dd9ddac (127.8.57.61:38809), 80c33542f22c408d9754bff6c9b7b89b (127.8.57.60:43497)
I20260929 02:55:15.499091 22029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7afa77707e34496092088221889988f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c33542f22c408d9754bff6c9b7b89b" is_pre_election: true
I20260929 02:55:15.499679 22029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7afa77707e34496092088221889988f2 in term 0.
I20260929 02:55:15.500348 21857 leader_election.cc:302] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7afa77707e34496092088221889988f2, 80c33542f22c408d9754bff6c9b7b89b; no voters:
I20260929 02:55:15.500860 22063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:55:15.500986 22063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:55:15.501080 22063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:15.501183 21980 log.cc:824] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:15.504298 22063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.505889 22029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7afa77707e34496092088221889988f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c33542f22c408d9754bff6c9b7b89b"
I20260929 02:55:15.506094 22029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:15.509248 22029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7afa77707e34496092088221889988f2 in term 1.
I20260929 02:55:15.509670 21857 leader_election.cc:302] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7afa77707e34496092088221889988f2, 80c33542f22c408d9754bff6c9b7b89b; no voters:
I20260929 02:55:15.510382 22063 leader_election.cc:288] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [CANDIDATE]: Term 1 election: Requested vote from peers 43b387c041154046bf5168811dd9ddac (127.8.57.61:38809), 80c33542f22c408d9754bff6c9b7b89b (127.8.57.60:43497)
I20260929 02:55:15.510591 22063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:15.511260 22063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [term 1 LEADER]: Becoming Leader. State: Replica: 7afa77707e34496092088221889988f2, State: Running, Role: LEADER
I20260929 02:55:15.511814 22063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.529762 22098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7afa77707e34496092088221889988f2. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.530159 22098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:15.531337 22100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:15.533313 21980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: No bootstrap required, opened a new log
W20260929 02:55:15.534343 21860 tablet.cc:2369] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:15.536746 22100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:15.543910 21980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.544286 21980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:15.544387 21980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b387c041154046bf5168811dd9ddac, State: Initialized, Role: FOLLOWER
I20260929 02:55:15.545077 21980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } }
I20260929 02:55:15.551965 22101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.552338 22101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:55:15.554121 21980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:15.554175 21955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7afa77707e34496092088221889988f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43b387c041154046bf5168811dd9ddac" is_pre_election: true
I20260929 02:55:15.554764 21955 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7afa77707e34496092088221889988f2 in term 0.
I20260929 02:55:15.555130 21954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7afa77707e34496092088221889988f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43b387c041154046bf5168811dd9ddac"
I20260929 02:55:15.555375 21954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:15.558640 21954 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7afa77707e34496092088221889988f2 in term 1.
I20260929 02:55:15.564240 22029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [term 1 FOLLOWER]: Refusing update from remote peer 7afa77707e34496092088221889988f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:15.565253 22098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:15.574826 21954 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [term 1 FOLLOWER]: Refusing update from remote peer 7afa77707e34496092088221889988f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:15.575752 22098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:15.604357 22101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7afa77707e34496092088221889988f2. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.604660 22101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:55:15.629328 22081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7afa77707e34496092088221889988f2. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.629670 22081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:55:15.631481 22101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.631742 22101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:55:15.635533 22100 catalog_manager.cc:1347] Generated new cluster ID: 1c8794930d35467fa0d1e904b1facff1
I20260929 02:55:15.635633 22100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:15.641398 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.641760 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:15.642149 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.642390 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:15.656471 22081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7afa77707e34496092088221889988f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7afa77707e34496092088221889988f2" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37569 } } peers { permanent_uuid: "43b387c041154046bf5168811dd9ddac" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 38809 } } peers { permanent_uuid: "80c33542f22c408d9754bff6c9b7b89b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 43497 } } }
I20260929 02:55:15.656750 22081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b [sys.catalog]: This master's current role is: FOLLOWER
W20260929 02:55:15.661705 22118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:55:15.661836 22118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:55:15.675002 22100 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:55:15.676761 22100 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:15.695994 22100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7afa77707e34496092088221889988f2: Generated new TSK 0
I20260929 02:55:15.700503 22100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:55:16.380456 22095 catalog_manager.cc:1259] Loaded cluster ID: 1c8794930d35467fa0d1e904b1facff1
I20260929 02:55:16.380551 22095 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: loading cluster ID for follower catalog manager: success
I20260929 02:55:16.382851 22095 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: acquiring CA information for follower catalog manager: success
I20260929 02:55:16.384600 22095 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80c33542f22c408d9754bff6c9b7b89b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:55:16.663698 22118 catalog_manager.cc:1259] Loaded cluster ID: 1c8794930d35467fa0d1e904b1facff1
I20260929 02:55:16.663795 22118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: loading cluster ID for follower catalog manager: success
I20260929 02:55:16.666206 22118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: acquiring CA information for follower catalog manager: success
I20260929 02:55:16.667932 22118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43b387c041154046bf5168811dd9ddac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:55:16.823091 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21839
I20260929 02:55:16.824213 21901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:16.933501 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21839
I20260929 02:55:16.949914 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21913
I20260929 02:55:16.950995 21975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:17.054596 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21913
I20260929 02:55:17.070071 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 21987
I20260929 02:55:17.071136 22049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:17.179994 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 21987
2026-09-29T02:55:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4497 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:55:17 dist-test-slave-1dkt krb5kdc[22132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:55:17 dist-test-slave-1dkt krb5kdc[22132](info): set up 2 sockets
Sep 29 02:55:17 dist-test-slave-1dkt krb5kdc[22132](info): commencing operation
krb5kdc: starting...
W20260929 02:55:18.176225 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:55:18 dist-test-slave-1dkt krb5kdc[22132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:55:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:55:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:55:18.382747 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37481
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:18.686255 22145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:18.686574 22145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:18.686712 22145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:18.693707 22145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:18.693809 22145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:18.693886 22145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:18.693982 22145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:18.694041 22145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:18.703302 22145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:37481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:18.704491 22145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:18.705750 22145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:18.717288 22150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:18.717272 22151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:18.717772 22153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:18.718264 22145 server_base.cc:1034] running on GCE node
I20260929 02:55:18.718972 22145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:18.720320 22145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:18.721521 22145 hybrid_clock.cc:648] HybridClock initialized: now 1790650518721469 us; error 34 us; skew 500 ppm
I20260929 02:55:18.721824 22145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:18 dist-test-slave-1dkt krb5kdc[22132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650518, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:18.732023 22145 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:18.733384 22145 webserver.cc:460] Webserver started at http://127.8.57.62:38891/ using document root <none> and password file <none>
I20260929 02:55:18.734028 22145 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:18.734135 22145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:18.734408 22145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:18.737236 22145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "8f6468f073e94372b9b206383cff70a7"
format_stamp: "Formatted at 2026-09-29 02:55:18 on dist-test-slave-1dkt"
server_key: "b622ae04eb9b1417ff8fafb5858800e2"
server_key_iv: "b6f220322728477c2e09fe42022a5dd3"
server_key_version: "encryptionkey@0"
I20260929 02:55:18.737931 22145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "8f6468f073e94372b9b206383cff70a7"
format_stamp: "Formatted at 2026-09-29 02:55:18 on dist-test-slave-1dkt"
server_key: "b622ae04eb9b1417ff8fafb5858800e2"
server_key_iv: "b6f220322728477c2e09fe42022a5dd3"
server_key_version: "encryptionkey@0"
I20260929 02:55:18.742592 22145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:55:18.745664 22160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:18.746886 22145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:55:18.747031 22145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "8f6468f073e94372b9b206383cff70a7"
format_stamp: "Formatted at 2026-09-29 02:55:18 on dist-test-slave-1dkt"
server_key: "b622ae04eb9b1417ff8fafb5858800e2"
server_key_iv: "b6f220322728477c2e09fe42022a5dd3"
server_key_version: "encryptionkey@0"
I20260929 02:55:18.747195 22145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:18.759483 22145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:18.763166 22145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:18.763404 22145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:18.779522 22145 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:37481
I20260929 02:55:18.779544 22211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:37481 every 8 connection(s)
I20260929 02:55:18.780841 22145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:18.784174 22212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:18.790452 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22145
I20260929 02:55:18.790637 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:55:18.791265 8420 external_mini_cluster.cc:1428] Setting key 9c08842ec1b13e3dd5a5859fafa22ac8
I20260929 02:55:18.794039 22212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7: Bootstrap starting.
I20260929 02:55:18.797230 22212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:18.798404 22212 log.cc:824] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:18.801370 22212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7: No bootstrap required, opened a new log
Sep 29 02:55:18 dist-test-slave-1dkt krb5kdc[22132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:18.810168 22212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6468f073e94372b9b206383cff70a7" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37481 } }
I20260929 02:55:18.810645 22212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:18.810745 22212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6468f073e94372b9b206383cff70a7, State: Initialized, Role: FOLLOWER
I20260929 02:55:18.811390 22212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6468f073e94372b9b206383cff70a7" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37481 } }
I20260929 02:55:18.811664 22212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:18.811803 22212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:55:18.811991 22212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:18.814670 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:18.792958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.8.57.62:37481)
0929 02:55:18.793720 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:18.793731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:18.793771 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:55:18.795479 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:18.795501 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:18.795907 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:18.796219 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:18.798073 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:18.799671 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:18.801502 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:18.802167 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:18.802297 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:18.807106 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:18.807116 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:18.807131 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:18.807165 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:18.810073 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:18.810949 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:18.810956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:18.810964 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:18.811200 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:18.811546 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:18.811550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:18.811553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:18.812296 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:18.812580 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:18.812925 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 02:55:18.813383 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":343,"threads_started":1}
I20260929 02:55:18.815122 22212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6468f073e94372b9b206383cff70a7" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37481 } }
I20260929 02:55:18.815578 22212 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6468f073e94372b9b206383cff70a7; no voters:
I20260929 02:55:18.816149 22212 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:55:18.816228 22217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:18.817641 22217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [term 1 LEADER]: Becoming Leader. State: Replica: 8f6468f073e94372b9b206383cff70a7, State: Running, Role: LEADER
I20260929 02:55:18.818143 22212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:18.818236 22217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6468f073e94372b9b206383cff70a7" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37481 } }
I20260929 02:55:18.818770 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:55:18.820729 22219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6468f073e94372b9b206383cff70a7. Latest consensus state: current_term: 1 leader_uuid: "8f6468f073e94372b9b206383cff70a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6468f073e94372b9b206383cff70a7" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37481 } } }
I20260929 02:55:18.821194 22219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:18.821766 22218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6468f073e94372b9b206383cff70a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6468f073e94372b9b206383cff70a7" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 37481 } } }
I20260929 02:55:18.822041 22218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:18.824138 22223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:18.827979 22223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:18.836489 22223 catalog_manager.cc:1347] Generated new cluster ID: 05e42faace3e4f8e9b7aef9606eea390
I20260929 02:55:18.836608 22223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:18.848208 22223 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:55:18.849424 22223 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:18.858771 22223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6468f073e94372b9b206383cff70a7: Generated new TSK 0
I20260929 02:55:18.859545 22223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:55:18.879508 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22145
I20260929 02:55:18.880510 22207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:18.971501 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22145
2026-09-29T02:55:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1788 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:55:19 dist-test-slave-1dkt krb5kdc[22240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:55:19 dist-test-slave-1dkt krb5kdc[22240](info): set up 2 sockets
Sep 29 02:55:19 dist-test-slave-1dkt krb5kdc[22240](info): commencing operation
krb5kdc: starting...
W20260929 02:55:19.960481 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:55:20 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:55:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:55:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:55:20.168283 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42753
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:20.473332 22253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:20.473613 22253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:20.473742 22253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:20.480733 22253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:20.480839 22253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:20.480914 22253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:20.480988 22253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:20.481045 22253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:20.490720 22253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:20.492095 22253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:20.493425 22253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:20.504470 22259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:20.504555 22258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:20.504527 22261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:20.505407 22253 server_base.cc:1034] running on GCE node
I20260929 02:55:20.506175 22253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:20.507529 22253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:20.508720 22253 hybrid_clock.cc:648] HybridClock initialized: now 1790650520508674 us; error 39 us; skew 500 ppm
I20260929 02:55:20.509083 22253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:20 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650520, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:20.512075 22253 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:20.513435 22253 webserver.cc:460] Webserver started at http://127.8.57.62:39067/ using document root <none> and password file <none>
I20260929 02:55:20.514081 22253 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:20.514192 22253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:20.514457 22253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:20.517309 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "1c6a62e7544f48a792e0cde7dbb0dd17"
format_stamp: "Formatted at 2026-09-29 02:55:20 on dist-test-slave-1dkt"
server_key: "642f9003bdf9ec4ca8feb3e1a05dc4ec"
server_key_iv: "7c76ed8e0f9dcfddaa5b297815fc4ea6"
server_key_version: "encryptionkey@0"
I20260929 02:55:20.518000 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "1c6a62e7544f48a792e0cde7dbb0dd17"
format_stamp: "Formatted at 2026-09-29 02:55:20 on dist-test-slave-1dkt"
server_key: "642f9003bdf9ec4ca8feb3e1a05dc4ec"
server_key_iv: "7c76ed8e0f9dcfddaa5b297815fc4ea6"
server_key_version: "encryptionkey@0"
I20260929 02:55:20.522696 22253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:55:20.525815 22268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:20.527033 22253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:55:20.527191 22253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "1c6a62e7544f48a792e0cde7dbb0dd17"
format_stamp: "Formatted at 2026-09-29 02:55:20 on dist-test-slave-1dkt"
server_key: "642f9003bdf9ec4ca8feb3e1a05dc4ec"
server_key_iv: "7c76ed8e0f9dcfddaa5b297815fc4ea6"
server_key_version: "encryptionkey@0"
I20260929 02:55:20.527350 22253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:20.547900 22253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:20.551944 22253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:20.552199 22253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:20.568909 22253 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42753
I20260929 02:55:20.568928 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42753 every 8 connection(s)
I20260929 02:55:20.570214 22253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:20.573832 22320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:20.575224 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22253
I20260929 02:55:20.575425 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:55:20.576355 8420 external_mini_cluster.cc:1428] Setting key 4e05ba2997d3c66682d499cb8a77eec6
I20260929 02:55:20.587850 22320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap starting.
Sep 29 02:55:20 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:20.591480 22320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:20.592638 22320 log.cc:824] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:20.595714 22320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: No bootstrap required, opened a new log
I20260929 02:55:20.599442 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:20.578592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.8.57.62:42753)
0929 02:55:20.579394 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:20.579405 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:20.579445 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:55:20.581035 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:20.581059 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:20.581443 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:20.581753 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:20.583442 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:20.584804 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:20.586653 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:20.587328 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:20.587463 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:20.592197 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:20.592204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:20.592219 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:20.592257 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:20.595318 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:20.596240 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:20.596247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:20.596253 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:20.596422 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:20.596836 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:20.596839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:20.596841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:20.597369 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:20.597601 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:20.598049 (+ 448us) server_negotiation.cc:299] Negotiation successful
0929 02:55:20.598400 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":375,"threads_started":1}
I20260929 02:55:20.603668 22320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:20.604032 22320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:20.604122 22320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c6a62e7544f48a792e0cde7dbb0dd17, State: Initialized, Role: FOLLOWER
I20260929 02:55:20.604617 22320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:20.604811 22320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:20.604933 22320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:55:20.605077 22320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:20.607136 22320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:20.607586 22320 leader_election.cc:302] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c6a62e7544f48a792e0cde7dbb0dd17; no voters:
I20260929 02:55:20.608258 22320 leader_election.cc:288] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:55:20.608368 22325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:20.609004 22325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 LEADER]: Becoming Leader. State: Replica: 1c6a62e7544f48a792e0cde7dbb0dd17, State: Running, Role: LEADER
I20260929 02:55:20.609541 22325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:20.610409 22320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:20.612388 22326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } } }
I20260929 02:55:20.612417 22327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c6a62e7544f48a792e0cde7dbb0dd17. Latest consensus state: current_term: 1 leader_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } } }
I20260929 02:55:20.612777 22327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:20.612777 22326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:20.615371 22330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:20.620508 22330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:20.627979 22330 catalog_manager.cc:1347] Generated new cluster ID: 9e2fe184c00545b2991acffe828834d2
I20260929 02:55:20.628094 22330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:20.659917 22330 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:55:20.661773 22330 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:20.669059 22330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Generated new TSK 0
I20260929 02:55:20.670083 22330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:55:20.725899 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42753
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:21.039492 22346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:21.039743 22346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:21.039866 22346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:21.047008 22346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:21.047107 22346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:21.047254 22346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:55:21.056886 22346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:21.058077 22346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:21.059348 22346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:21.071575 22352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:21.071645 22351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:21.071655 22354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:21.072649 22346 server_base.cc:1034] running on GCE node
I20260929 02:55:21.073243 22346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:21.074110 22346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:21.075296 22346 hybrid_clock.cc:648] HybridClock initialized: now 1790650521075257 us; error 35 us; skew 500 ppm
I20260929 02:55:21.075577 22346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:21 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:21.078478 22346 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:55:21.079764 22346 webserver.cc:460] Webserver started at http://127.8.57.1:46595/ using document root <none> and password file <none>
I20260929 02:55:21.080413 22346 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:21.080514 22346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:21.080776 22346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:21.083500 22346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "248996eb1a4e4601867557174b158383"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "2ea48f476d2bb9a46ea0501abcb373c1"
server_key_iv: "056bc46ca97f44b087d7e61d60fafde2"
server_key_version: "encryptionkey@0"
I20260929 02:55:21.084218 22346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "248996eb1a4e4601867557174b158383"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "2ea48f476d2bb9a46ea0501abcb373c1"
server_key_iv: "056bc46ca97f44b087d7e61d60fafde2"
server_key_version: "encryptionkey@0"
I20260929 02:55:21.088900 22346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:55:21.092036 22361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:21.093241 22346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:55:21.093367 22346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "248996eb1a4e4601867557174b158383"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "2ea48f476d2bb9a46ea0501abcb373c1"
server_key_iv: "056bc46ca97f44b087d7e61d60fafde2"
server_key_version: "encryptionkey@0"
I20260929 02:55:21.093528 22346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:21.127208 22346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:21.130990 22346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:21.131274 22346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:21.132346 22346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:21.134709 22346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:21.134799 22346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:21.134919 22346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:21.134977 22346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:21.164762 22346 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:37399
I20260929 02:55:21.164801 22473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:37399 every 8 connection(s)
I20260929 02:55:21.165931 22346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:55:21.167989 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22346
I20260929 02:55:21.168190 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:55:21.168977 8420 external_mini_cluster.cc:1428] Setting key 048ea56d4701938e448a7a30969959eb
Sep 29 02:55:21 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:21.184568 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:21.168510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:60251 (local address 127.8.57.62:42753)
0929 02:55:21.168885 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:21.168890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:21.169906 (+ 1016us) server_negotiation.cc:407] Connection header received
0929 02:55:21.171940 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:21.171946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:21.172031 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:21.172124 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:21.173527 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.174216 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.175690 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.176131 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.176236 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:21.179351 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:21.179355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:21.179358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:21.179389 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:21.181755 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:21.182420 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:21.182426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:21.182428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:21.182545 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:21.182933 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:21.182937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:21.182940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:21.183203 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:21.183323 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:21.184002 (+ 679us) server_negotiation.cc:299] Negotiation successful
0929 02:55:21.184222 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":188,"threads_started":1}
I20260929 02:55:21.185421 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:21.168997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:60251)
0929 02:55:21.169726 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:21.169779 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:21.171678 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:21.172363 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:21.172371 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:21.172834 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:21.173363 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:21.173381 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.174353 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.174357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:21.175555 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:21.175567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.176330 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.176337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:21.176548 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:21.177137 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:21.177163 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:21.179156 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:21.181891 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:21.181899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:21.181915 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:21.182308 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:21.182653 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:21.182658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:21.182662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:21.182833 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:21.183328 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:21.183336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:21.183637 (+ 301us) client_negotiation.cc:769] Sending connection context
0929 02:55:21.183860 (+ 223us) client_negotiation.cc:240] Negotiation successful
0929 02:55:21.184357 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":287,"threads_started":1}
I20260929 02:55:21.186827 22474 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42753
I20260929 02:55:21.187120 22474 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
I20260929 02:55:21.187765 22474 heartbeater.cc:511] Master 127.8.57.62:42753 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
I20260929 02:55:21.189838 22285 ts_manager.cc:194] Registered new tserver with Master: 248996eb1a4e4601867557174b158383 (127.8.57.1:37399)
I20260929 02:55:21.191195 22285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:60251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:55:21.212864 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42753
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:21.513995 22480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:21.514257 22480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:21.514384 22480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:21.521363 22480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:21.521472 22480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:21.521629 22480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:55:21.530925 22480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:21.532102 22480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:21.533334 22480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:21.544960 22488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:21.544999 22486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:21.545163 22485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:21.545866 22480 server_base.cc:1034] running on GCE node
I20260929 02:55:21.546415 22480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:21.547261 22480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:21.548442 22480 hybrid_clock.cc:648] HybridClock initialized: now 1790650521548398 us; error 32 us; skew 500 ppm
I20260929 02:55:21.548744 22480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:21 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:21.551517 22480 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:55:21.552814 22480 webserver.cc:460] Webserver started at http://127.8.57.2:33467/ using document root <none> and password file <none>
I20260929 02:55:21.553431 22480 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:21.553537 22480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:21.553803 22480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:21.556615 22480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "e7b471895f9045d18c2c59fadee54d86"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "8c9e1f87a4c188098869efe69863117a"
server_key_iv: "4665fdecc8e22f376964daeb2df9aeeb"
server_key_version: "encryptionkey@0"
I20260929 02:55:21.557343 22480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "e7b471895f9045d18c2c59fadee54d86"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "8c9e1f87a4c188098869efe69863117a"
server_key_iv: "4665fdecc8e22f376964daeb2df9aeeb"
server_key_version: "encryptionkey@0"
I20260929 02:55:21.562207 22480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 02:55:21.565395 22495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:21.566610 22480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:55:21.566748 22480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "e7b471895f9045d18c2c59fadee54d86"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "8c9e1f87a4c188098869efe69863117a"
server_key_iv: "4665fdecc8e22f376964daeb2df9aeeb"
server_key_version: "encryptionkey@0"
I20260929 02:55:21.566936 22480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:21.583122 22480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:21.586567 22480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:21.586814 22480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:21.587710 22480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:21.588966 22480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:21.589040 22480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:21.589154 22480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:21.589208 22480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:21.618000 22480 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:37927
I20260929 02:55:21.618047 22607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:37927 every 8 connection(s)
I20260929 02:55:21.619212 22480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:55:21.620744 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22480
I20260929 02:55:21.620890 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:55:21.621448 8420 external_mini_cluster.cc:1428] Setting key a6b435ad8eeba223a243c5ccb2493b50
Sep 29 02:55:21 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:21.640671 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:21.621742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:40895 (local address 127.8.57.62:42753)
0929 02:55:21.621908 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:21.621912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:21.623215 (+ 1303us) server_negotiation.cc:407] Connection header received
0929 02:55:21.625662 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:21.625666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:21.625732 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:21.625815 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:21.627695 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.628435 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.630466 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.630879 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.630992 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:21.634888 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:21.634894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:21.634899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:21.634938 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:21.637823 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:21.638487 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:21.638492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:21.638495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:21.638663 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:21.639010 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:21.639014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:21.639017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:21.639338 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:21.639466 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:21.640091 (+ 625us) server_negotiation.cc:299] Negotiation successful
0929 02:55:21.640316 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:55:21.641492 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:21.622286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:40895)
0929 02:55:21.623059 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:21.623101 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:21.625369 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:21.626072 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:21.626084 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:21.626746 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:21.627536 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:21.627560 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.628567 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.628572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:21.630346 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:21.630360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:21.631100 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:21.631107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:21.631385 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:21.632147 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:21.632182 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:21.634653 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:21.637960 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:21.637967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:21.637980 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:21.638386 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:21.638754 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:21.638757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:21.638759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:21.638923 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:21.639443 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:21.639449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:21.639749 (+ 300us) client_negotiation.cc:769] Sending connection context
0929 02:55:21.640000 (+ 251us) client_negotiation.cc:240] Negotiation successful
0929 02:55:21.640392 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":342,"threads_started":1}
I20260929 02:55:21.643002 22608 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42753
I20260929 02:55:21.643276 22608 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:21.643992 22608 heartbeater.cc:511] Master 127.8.57.62:42753 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
I20260929 02:55:21.645411 22285 ts_manager.cc:194] Registered new tserver with Master: e7b471895f9045d18c2c59fadee54d86 (127.8.57.2:37927)
I20260929 02:55:21.646291 22285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:40895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:55:21.671048 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42753
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:21.978832 22613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:21.979164 22613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:21.979310 22613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:21.986346 22613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:21.986446 22613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:21.986603 22613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:55:21.996325 22613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:21.997602 22613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:21.998885 22613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:22.011672 22618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:22.011650 22619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:22.011945 22621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:22.012706 22613 server_base.cc:1034] running on GCE node
I20260929 02:55:22.013298 22613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:22.014261 22613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:22.015465 22613 hybrid_clock.cc:648] HybridClock initialized: now 1790650522015393 us; error 62 us; skew 500 ppm
I20260929 02:55:22.015794 22613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:22 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:22.018819 22613 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:55:22.020329 22613 webserver.cc:460] Webserver started at http://127.8.57.3:40885/ using document root <none> and password file <none>
I20260929 02:55:22.021039 22613 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:22.021152 22613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:22.021428 22613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:22.024281 22613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "8a8418509cbf4362897529eda3777065"
format_stamp: "Formatted at 2026-09-29 02:55:22 on dist-test-slave-1dkt"
server_key: "acb0a433b8d8ed943f3cbbacd95b612d"
server_key_iv: "5638e9b715665ca565c05596fd690f02"
server_key_version: "encryptionkey@0"
I20260929 02:55:22.024993 22613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "8a8418509cbf4362897529eda3777065"
format_stamp: "Formatted at 2026-09-29 02:55:22 on dist-test-slave-1dkt"
server_key: "acb0a433b8d8ed943f3cbbacd95b612d"
server_key_iv: "5638e9b715665ca565c05596fd690f02"
server_key_version: "encryptionkey@0"
I20260929 02:55:22.029978 22613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:55:22.033567 22628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:22.034850 22613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:55:22.034991 22613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "8a8418509cbf4362897529eda3777065"
format_stamp: "Formatted at 2026-09-29 02:55:22 on dist-test-slave-1dkt"
server_key: "acb0a433b8d8ed943f3cbbacd95b612d"
server_key_iv: "5638e9b715665ca565c05596fd690f02"
server_key_version: "encryptionkey@0"
I20260929 02:55:22.035173 22613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:22.065259 22613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:22.069468 22613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:22.069775 22613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:22.070760 22613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:22.072181 22613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:22.072264 22613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:22.072376 22613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:22.072435 22613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:22.102236 22613 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:45491
I20260929 02:55:22.102268 22740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:45491 every 8 connection(s)
I20260929 02:55:22.103446 22613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:55:22.111362 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22613
I20260929 02:55:22.111502 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:55:22.112113 8420 external_mini_cluster.cc:1428] Setting key 869a8e1992f2c7be15169186f3714b07
Sep 29 02:55:22 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:22.122229 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.105883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:41115 (local address 127.8.57.62:42753)
0929 02:55:22.106070 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.106074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.107175 (+ 1101us) server_negotiation.cc:407] Connection header received
0929 02:55:22.108999 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.109005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.109093 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.109179 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:22.110668 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.111428 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.113485 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.113962 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.114093 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.117383 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:22.117388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:22.117392 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:22.117425 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:22.119694 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.120389 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.120392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.120395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.120505 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.120876 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.120878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.120880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.121143 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:22.121249 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.121812 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.121975 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:55:22.123162 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.106355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:41115)
0929 02:55:22.107028 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:22.107069 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:22.108734 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:22.109412 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:22.109420 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:22.109899 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:22.110439 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.110457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.111615 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.111620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.113356 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.113373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.114149 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.114153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.114342 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.115015 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:22.115044 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:22.117213 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:22.119836 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:22.119842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:22.119854 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:22.120293 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:22.120580 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:22.120584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:22.120586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:22.120797 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:22.121230 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:22.121235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:22.121526 (+ 291us) client_negotiation.cc:769] Sending connection context
0929 02:55:22.121731 (+ 205us) client_negotiation.cc:240] Negotiation successful
0929 02:55:22.122106 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":263,"threads_started":1}
I20260929 02:55:22.124533 22741 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42753
I20260929 02:55:22.124812 22741 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:22.125473 22741 heartbeater.cc:511] Master 127.8.57.62:42753 requested a full tablet report, sending...
I20260929 02:55:22.126919 22284 ts_manager.cc:194] Registered new tserver with Master: 8a8418509cbf4362897529eda3777065 (127.8.57.3:45491)
I20260929 02:55:22.127763 22284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:41115
I20260929 02:55:22.139268 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:55:22.155292 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.143538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.8.57.62:42753)
0929 02:55:22.143693 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.143699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.144109 (+ 410us) server_negotiation.cc:407] Connection header received
0929 02:55:22.144280 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.144284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.144345 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.144423 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:22.145711 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.146417 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.147737 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.148177 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.148282 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.150477 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:22.150482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:22.150485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:22.150520 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:22.152758 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.153396 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.153401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.153406 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.153537 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.153895 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.153900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.153904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.154221 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:22.154454 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.154768 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.154983 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:55:22.159134 22284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:55:22.166175 22284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:55:22.189232 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.177844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:37927 (local address 127.0.0.1:52192)
0929 02:55:22.178914 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:22.178951 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:22.179137 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:22.180191 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:22.180199 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:22.180233 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:22.180632 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.180644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.182581 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.182588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.185543 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.185561 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.188238 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.188246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.188471 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.188529 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 02:55:22.188635 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 02:55:22.188839 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":761,"threads_started":1}
I20260929 02:55:22.191636 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.178189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45491 (local address 127.0.0.1:53820)
0929 02:55:22.181193 (+ 3004us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:22.181211 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:22.181314 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:22.182628 (+ 1314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:22.182632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:22.182661 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:22.183816 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.183828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.186484 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.186490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.188772 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.188788 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.190860 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.190865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.191027 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.191056 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:55:22.191134 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:55:22.191254 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2849,"thread_start_us":2723,"threads_started":1}
I20260929 02:55:22.191692 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.178428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.8.57.3:45491)
0929 02:55:22.181822 (+ 3394us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.181830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.181861 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:55:22.181946 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.181954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.182179 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.182351 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:22.184103 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.186295 (+ 2192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.188961 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.190382 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.190504 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.190739 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.191219 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.191383 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3251,"thread_start_us":3111,"threads_started":1}
I20260929 02:55:22.191670 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.178016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.8.57.2:37927)
0929 02:55:22.178574 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.178600 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.179065 (+ 465us) server_negotiation.cc:407] Connection header received
0929 02:55:22.179426 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.179436 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.179697 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.179901 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:22.180828 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.182416 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.185749 (+ 3333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.188001 (+ 2252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.190960 (+ 2959us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.191164 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.191224 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.191379 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":280,"threads_started":1}
I20260929 02:55:22.194224 22474 heartbeater.cc:503] Master 127.8.57.62:42753 was elected leader, sending a full tablet report...
I20260929 02:55:22.195346 22543 tablet_service.cc:1467] Processing CreateTablet for tablet e0bb1dd1ae74414aa9dcc41968c16035 (DEFAULT_TABLE table=test-table [id=2efae70da9ff4181bea857b28a5b98e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:22.195345 22676 tablet_service.cc:1467] Processing CreateTablet for tablet e0bb1dd1ae74414aa9dcc41968c16035 (DEFAULT_TABLE table=test-table [id=2efae70da9ff4181bea857b28a5b98e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:22.196846 22543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bb1dd1ae74414aa9dcc41968c16035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:22.196882 22676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bb1dd1ae74414aa9dcc41968c16035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:22.198618 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.181304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52364)
0929 02:55:22.189726 (+ 8422us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:22.189744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:22.189838 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:22.192126 (+ 2288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:22.192130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:22.192156 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:22.192297 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.192307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.194391 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.194396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.196211 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.196225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.197890 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.197896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.198105 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.198138 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:55:22.198231 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 02:55:22.198366 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8306,"thread_start_us":10501,"threads_started":1}
I20260929 02:55:22.198895 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.181991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.8.57.1:37399)
0929 02:55:22.190927 (+ 8936us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.190933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.190956 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:55:22.191032 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.191037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.191236 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.192469 (+ 1233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:22.192709 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.194230 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.196381 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.197739 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.197865 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.198066 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.198283 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.198438 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8779,"thread_start_us":8617,"threads_started":1}
I20260929 02:55:22.202426 22409 tablet_service.cc:1467] Processing CreateTablet for tablet e0bb1dd1ae74414aa9dcc41968c16035 (DEFAULT_TABLE table=test-table [id=2efae70da9ff4181bea857b28a5b98e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:22.203788 22409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bb1dd1ae74414aa9dcc41968c16035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:22.208341 22759 tablet_bootstrap.cc:492] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Bootstrap starting.
I20260929 02:55:22.210248 22760 tablet_bootstrap.cc:492] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Bootstrap starting.
I20260929 02:55:22.213376 22760 tablet_bootstrap.cc:654] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:22.213376 22759 tablet_bootstrap.cc:654] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:22.214586 22759 log.cc:824] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:22.214569 22760 log.cc:824] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:22.216948 22763 tablet_bootstrap.cc:492] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Bootstrap starting.
I20260929 02:55:22.217594 22759 tablet_bootstrap.cc:492] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: No bootstrap required, opened a new log
I20260929 02:55:22.217674 22760 tablet_bootstrap.cc:492] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: No bootstrap required, opened a new log
I20260929 02:55:22.217854 22759 ts_tablet_manager.cc:1397] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260929 02:55:22.217926 22760 ts_tablet_manager.cc:1397] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:55:22.220986 22763 tablet_bootstrap.cc:654] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:22.222211 22763 log.cc:824] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:22.225148 22763 tablet_bootstrap.cc:492] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: No bootstrap required, opened a new log
I20260929 02:55:22.225389 22763 ts_tablet_manager.cc:1397] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260929 02:55:22.225244 22759 raft_consensus.cc:348] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.225597 22759 raft_consensus.cc:374] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:22.225690 22759 raft_consensus.cc:729] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8418509cbf4362897529eda3777065, State: Initialized, Role: FOLLOWER
I20260929 02:55:22.226217 22759 consensus_queue.cc:260] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.227206 22741 heartbeater.cc:503] Master 127.8.57.62:42753 was elected leader, sending a full tablet report...
I20260929 02:55:22.227246 22760 raft_consensus.cc:348] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.227615 22760 raft_consensus.cc:374] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:22.227695 22760 raft_consensus.cc:729] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b471895f9045d18c2c59fadee54d86, State: Initialized, Role: FOLLOWER
I20260929 02:55:22.227846 22759 ts_tablet_manager.cc:1428] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260929 02:55:22.228313 22760 consensus_queue.cc:260] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.229311 22608 heartbeater.cc:503] Master 127.8.57.62:42753 was elected leader, sending a full tablet report...
I20260929 02:55:22.229889 22760 ts_tablet_manager.cc:1428] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260929 02:55:22.235284 22763 raft_consensus.cc:348] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.235617 22763 raft_consensus.cc:374] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:22.235708 22763 raft_consensus.cc:729] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 248996eb1a4e4601867557174b158383, State: Initialized, Role: FOLLOWER
I20260929 02:55:22.236243 22763 consensus_queue.cc:260] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.237694 22763 ts_tablet_manager.cc:1428] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260929 02:55:22.347523 22766 raft_consensus.cc:482] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:55:22.347795 22766 raft_consensus.cc:504] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.349352 22766 leader_election.cc:288] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a8418509cbf4362897529eda3777065 (127.8.57.3:45491), 248996eb1a4e4601867557174b158383 (127.8.57.1:37399)
W20260929 02:55:22.355000 22742 tablet.cc:2369] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:22.356278 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.350567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:43319 (local address 127.8.57.1:37399)
0929 02:55:22.350754 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.350760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.350990 (+ 230us) server_negotiation.cc:407] Connection header received
0929 02:55:22.351130 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.351135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.351209 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.351296 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:22.351840 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.352673 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.354344 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.355085 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.355200 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.355298 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.355903 (+ 605us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.356037 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:55:22.356272 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.349749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:32841 (local address 127.8.57.3:45491)
0929 02:55:22.349915 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.349920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.350324 (+ 404us) server_negotiation.cc:407] Connection header received
0929 02:55:22.350554 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.350560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.350645 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.350734 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:22.351411 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.352323 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.354209 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.355238 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.355358 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.355451 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.355912 (+ 461us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.356043 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:55:22.356496 22768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.349619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45491 (local address 127.8.57.2:32841)
0929 02:55:22.350224 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:22.350244 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:22.350382 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:22.350842 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:22.350847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:22.350878 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:22.351254 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.351268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.352449 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.352454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.354042 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.354055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.355440 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.355446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.355692 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.355745 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 02:55:22.355839 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 02:55:22.356058 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":105,"thread_start_us":274,"threads_started":1}
I20260929 02:55:22.356877 22769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.350328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.8.57.2:43319)
0929 02:55:22.350889 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:22.350905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:22.351017 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:22.351321 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:22.351325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:22.351348 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:22.351714 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.351726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.352804 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.352808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.354188 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:22.354202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.355409 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.355417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:22.355689 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.355759 (+ 70us) client_negotiation.cc:769] Sending connection context
0929 02:55:22.355839 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:55:22.356057 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":82,"thread_start_us":298,"threads_started":1}
I20260929 02:55:22.357474 22696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb1dd1ae74414aa9dcc41968c16035" candidate_uuid: "e7b471895f9045d18c2c59fadee54d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a8418509cbf4362897529eda3777065" is_pre_election: true
I20260929 02:55:22.357807 22429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb1dd1ae74414aa9dcc41968c16035" candidate_uuid: "e7b471895f9045d18c2c59fadee54d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "248996eb1a4e4601867557174b158383" is_pre_election: true
I20260929 02:55:22.357915 22696 raft_consensus.cc:2454] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7b471895f9045d18c2c59fadee54d86 in term 0.
I20260929 02:55:22.358225 22429 raft_consensus.cc:2454] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7b471895f9045d18c2c59fadee54d86 in term 0.
I20260929 02:55:22.358521 22497 leader_election.cc:302] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a8418509cbf4362897529eda3777065, e7b471895f9045d18c2c59fadee54d86; no voters:
I20260929 02:55:22.358872 22766 raft_consensus.cc:2781] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:55:22.359012 22766 raft_consensus.cc:482] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:55:22.359117 22766 raft_consensus.cc:3037] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:22.361294 22766 raft_consensus.cc:504] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.361837 22766 leader_election.cc:288] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [CANDIDATE]: Term 1 election: Requested vote from peers 8a8418509cbf4362897529eda3777065 (127.8.57.3:45491), 248996eb1a4e4601867557174b158383 (127.8.57.1:37399)
I20260929 02:55:22.362201 22696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb1dd1ae74414aa9dcc41968c16035" candidate_uuid: "e7b471895f9045d18c2c59fadee54d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a8418509cbf4362897529eda3777065"
I20260929 02:55:22.362295 22429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb1dd1ae74414aa9dcc41968c16035" candidate_uuid: "e7b471895f9045d18c2c59fadee54d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "248996eb1a4e4601867557174b158383"
I20260929 02:55:22.362370 22696 raft_consensus.cc:3037] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:22.362478 22429 raft_consensus.cc:3037] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:22.364550 22429 raft_consensus.cc:2454] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7b471895f9045d18c2c59fadee54d86 in term 1.
I20260929 02:55:22.364564 22696 raft_consensus.cc:2454] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7b471895f9045d18c2c59fadee54d86 in term 1.
I20260929 02:55:22.364904 22497 leader_election.cc:302] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 248996eb1a4e4601867557174b158383, e7b471895f9045d18c2c59fadee54d86; no voters:
I20260929 02:55:22.365183 22766 raft_consensus.cc:2781] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:22.365584 22766 raft_consensus.cc:686] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 1 LEADER]: Becoming Leader. State: Replica: e7b471895f9045d18c2c59fadee54d86, State: Running, Role: LEADER
I20260929 02:55:22.366009 22766 consensus_queue.cc:237] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } }
I20260929 02:55:22.370496 22285 catalog_manager.cc:5495] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7b471895f9045d18c2c59fadee54d86 (127.8.57.2). New cstate: current_term: 1 leader_uuid: "e7b471895f9045d18c2c59fadee54d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } health_report { overall_health: UNKNOWN } } }
W20260929 02:55:22.370941 22609 tablet.cc:2369] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:55:22.417985 22475 tablet.cc:2369] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:22.424782 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.419916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52206 (local address 127.8.57.2:37927)
0929 02:55:22.420101 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.420105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.420122 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:55:22.420232 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.420236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.420317 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.420428 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:55:22.420742 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.421450 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.422816 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.423215 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.423321 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.423748 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:55:22.424310 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:55:22.424433 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.424474 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.424539 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:55:22.432783 22429 raft_consensus.cc:1261] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Refusing update from remote peer e7b471895f9045d18c2c59fadee54d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:22.432790 22696 raft_consensus.cc:1261] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Refusing update from remote peer e7b471895f9045d18c2c59fadee54d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:22.433638 22770 consensus_queue.cc:1035] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:22.433975 22766 consensus_queue.cc:1035] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:22.443734 22773 mvcc.cc:205] Tried to move back new op lower bound from 7334504539874525184 to 7334504539614830592. Current Snapshot: MvccSnapshot[applied={T|T < 7334504539874525184}]
I20260929 02:55:22.458096 22284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:22.458343 22284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:22.461501 22284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Sending DeleteTablet for 3 replicas of tablet e0bb1dd1ae74414aa9dcc41968c16035
I20260929 02:55:22.462651 22543 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bb1dd1ae74414aa9dcc41968c16035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:22 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:52192
I20260929 02:55:22.462630 22676 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bb1dd1ae74414aa9dcc41968c16035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:22 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:53820
I20260929 02:55:22.463301 22409 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bb1dd1ae74414aa9dcc41968c16035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:22 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:52364
I20260929 02:55:22.463440 22784 tablet_replica.cc:330] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: stopping tablet replica
I20260929 02:55:22.463955 22784 raft_consensus.cc:2229] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:55:22.464475 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22346
I20260929 02:55:22.464663 22784 raft_consensus.cc:2258] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:22.465408 22783 tablet_replica.cc:330] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: stopping tablet replica
I20260929 02:55:22.465822 22783 raft_consensus.cc:2229] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:22.466408 22785 tablet_replica.cc:330] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: stopping tablet replica
I20260929 02:55:22.466718 22784 ts_tablet_manager.cc:1905] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:22.466758 22785 raft_consensus.cc:2229] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:22.467228 22785 raft_consensus.cc:2258] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:22.467849 22783 raft_consensus.cc:2258] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:22.469674 22783 ts_tablet_manager.cc:1905] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:22.469699 22785 ts_tablet_manager.cc:1905] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 02:55:22 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:55:22.476406 22783 ts_tablet_manager.cc:1918] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:22.476564 22783 log.cc:1196] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/e0bb1dd1ae74414aa9dcc41968c16035
I20260929 02:55:22.476943 22785 ts_tablet_manager.cc:1918] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:22.477078 22783 ts_tablet_manager.cc:1939] T e0bb1dd1ae74414aa9dcc41968c16035 P 8a8418509cbf4362897529eda3777065: Deleting consensus metadata
I20260929 02:55:22.477546 22785 log.cc:1196] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/e0bb1dd1ae74414aa9dcc41968c16035
I20260929 02:55:22.478396 22785 ts_tablet_manager.cc:1939] T e0bb1dd1ae74414aa9dcc41968c16035 P 248996eb1a4e4601867557174b158383: Deleting consensus metadata
I20260929 02:55:22.478899 22270 catalog_manager.cc:4841] TS 8a8418509cbf4362897529eda3777065 (127.8.57.3:45491): tablet e0bb1dd1ae74414aa9dcc41968c16035 (table test-table [id=2efae70da9ff4181bea857b28a5b98e1]) successfully deleted
I20260929 02:55:22.480262 22270 catalog_manager.cc:4841] TS 248996eb1a4e4601867557174b158383 (127.8.57.1:37399): tablet e0bb1dd1ae74414aa9dcc41968c16035 (table test-table [id=2efae70da9ff4181bea857b28a5b98e1]) successfully deleted
I20260929 02:55:22.483098 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.465555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.8.57.1:37399)
0929 02:55:22.465709 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.465715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.465798 (+ 83us) server_negotiation.cc:407] Connection header received
0929 02:55:22.466062 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.466068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.466194 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.466264 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:22.467865 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.468964 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.470997 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.471666 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.471762 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.476949 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:22.476963 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:22.476980 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:22.477026 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:22.480556 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.481175 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.481183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.481185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.481307 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.481673 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.481678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.481681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.482006 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:22.482278 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.482551 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.482755 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:55:22.483680 22469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:22.484890 22784 ts_tablet_manager.cc:1918] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:22.485076 22784 log.cc:1196] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/e0bb1dd1ae74414aa9dcc41968c16035
I20260929 02:55:22.485801 22784 ts_tablet_manager.cc:1939] T e0bb1dd1ae74414aa9dcc41968c16035 P e7b471895f9045d18c2c59fadee54d86: Deleting consensus metadata
I20260929 02:55:22.487658 22269 catalog_manager.cc:4841] TS e7b471895f9045d18c2c59fadee54d86 (127.8.57.2:37927): tablet e0bb1dd1ae74414aa9dcc41968c16035 (table test-table [id=2efae70da9ff4181bea857b28a5b98e1]) successfully deleted
I20260929 02:55:22.571969 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22346
I20260929 02:55:22.593235 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22480
Sep 29 02:55:22 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:55:22.608237 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.593984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.8.57.2:37927)
0929 02:55:22.594169 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.594174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.594217 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:55:22.594416 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.594420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.594485 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.594571 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:22.595847 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.596788 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.598059 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.598502 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.598615 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.602578 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:22.602596 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:22.602610 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:22.602651 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:22.605519 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.606150 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.606157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.606163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.606297 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.606650 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.606654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.606656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.607059 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:22.607376 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.607665 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.607918 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:55:22.608961 22603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:22.718952 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22480
I20260929 02:55:22.733808 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22613
Sep 29 02:55:22 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:55:22.747442 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:22.734493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.8.57.3:45491)
0929 02:55:22.734663 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:22.734669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:22.734688 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:55:22.734858 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:22.734863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:22.734934 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:22.735015 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:22.736292 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.736990 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.738283 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:22.738703 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:22.738806 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:22.742931 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:22.742944 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:22.742958 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:22.742989 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:22.745300 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.745882 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.745887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.745889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.746006 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:22.746286 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:22.746289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:22.746291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:22.746605 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:22.746814 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:22.747044 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:55:22.747192 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:55:22.748095 22736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:22.833528 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22613
I20260929 02:55:22.846145 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22253
I20260929 02:55:22.847182 22315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:22.953953 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22253
I20260929 02:55:22.970453 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42753
--webserver_interface=127.8.57.62
--webserver_port=39067
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:23.286475 22791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:23.286762 22791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:23.286881 22791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:23.294142 22791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:23.294257 22791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:23.294312 22791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:23.294392 22791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:23.294454 22791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:23.304800 22791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=39067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:23.306123 22791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:23.307478 22791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:23.329378 22800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:23.329459 22798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:23.329409 22797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:23.333496 22791 server_base.cc:1034] running on GCE node
I20260929 02:55:23.334388 22791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:23.336054 22791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:23.337272 22791 hybrid_clock.cc:648] HybridClock initialized: now 1790650523337216 us; error 53 us; skew 500 ppm
I20260929 02:55:23.337661 22791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:23 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650523, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:23.341140 22791 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:23.342818 22791 webserver.cc:460] Webserver started at http://127.8.57.62:39067/ using document root <none> and password file <none>
I20260929 02:55:23.343576 22791 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:23.343707 22791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:23.349673 22791 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:55:23.352491 22807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:23.353782 22791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:23.353941 22791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "1c6a62e7544f48a792e0cde7dbb0dd17"
format_stamp: "Formatted at 2026-09-29 02:55:20 on dist-test-slave-1dkt"
server_key: "642f9003bdf9ec4ca8feb3e1a05dc4ec"
server_key_iv: "7c76ed8e0f9dcfddaa5b297815fc4ea6"
server_key_version: "encryptionkey@0"
I20260929 02:55:23.354972 22791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:23.377684 22791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:23.381899 22791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:23.382181 22791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:23.399530 22791 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42753
I20260929 02:55:23.399557 22858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42753 every 8 connection(s)
I20260929 02:55:23.400986 22791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:23.407689 22859 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:55:23.409266 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22791
I20260929 02:55:23.410198 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:37399
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:42753
--webserver_port=46595
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:23.411828 22859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap starting.
I20260929 02:55:23.432018 22859 log.cc:824] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:23.443503 22859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:55:23.444015 22859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap complete.
I20260929 02:55:23.452737 22859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:23.453523 22859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c6a62e7544f48a792e0cde7dbb0dd17, State: Initialized, Role: FOLLOWER
I20260929 02:55:23.454095 22859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:23.454313 22859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:23.454427 22859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:55:23.454566 22859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:55:23.458433 22859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:23.458952 22859 leader_election.cc:302] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c6a62e7544f48a792e0cde7dbb0dd17; no voters:
I20260929 02:55:23.459568 22859 leader_election.cc:288] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:55:23.459722 22863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:55:23.461823 22859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:23.466779 22863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 LEADER]: Becoming Leader. State: Replica: 1c6a62e7544f48a792e0cde7dbb0dd17, State: Running, Role: LEADER
I20260929 02:55:23.467592 22863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:23.470162 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } } }
I20260929 02:55:23.470562 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:23.471093 22865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c6a62e7544f48a792e0cde7dbb0dd17. Latest consensus state: current_term: 2 leader_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } } }
I20260929 02:55:23.471369 22865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:23.473383 22874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:23.480957 22874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:23.482475 22874 catalog_manager.cc:1259] Loaded cluster ID: 9e2fe184c00545b2991acffe828834d2
I20260929 02:55:23.482641 22874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:23.485896 22874 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 02:55:23.499382 22874 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 02:55:23.506125 22874 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:23.507438 22874 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Loaded TSK: 0
I20260929 02:55:23.508569 22874 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:55:23.753841 22861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:23.754101 22861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:23.754217 22861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:23.761435 22861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:23.761531 22861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:23.761668 22861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:55:23.771255 22861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:37399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=46595
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:23.772465 22861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:23.773679 22861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:23.787802 22890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:23.787829 22888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:23.788456 22887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:23.789201 22861 server_base.cc:1034] running on GCE node
I20260929 02:55:23.789733 22861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:23.790621 22861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:23.791811 22861 hybrid_clock.cc:648] HybridClock initialized: now 1790650523791747 us; error 62 us; skew 500 ppm
I20260929 02:55:23.792114 22861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:23 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650523, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:23.798666 22861 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:55:23.800122 22861 webserver.cc:460] Webserver started at http://127.8.57.1:46595/ using document root <none> and password file <none>
I20260929 02:55:23.800786 22861 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:23.800918 22861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:23.806411 22861 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260929 02:55:23.809197 22897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:23.810586 22861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:23.810747 22861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "248996eb1a4e4601867557174b158383"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "2ea48f476d2bb9a46ea0501abcb373c1"
server_key_iv: "056bc46ca97f44b087d7e61d60fafde2"
server_key_version: "encryptionkey@0"
I20260929 02:55:23.811702 22861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:23.833169 22861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:23.837205 22861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:23.837438 22861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:23.838387 22861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:23.839726 22861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:23.839816 22861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:23.839939 22861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:23.840006 22861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:23.869585 22861 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:37399
I20260929 02:55:23.869642 23009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:37399 every 8 connection(s)
I20260929 02:55:23.870958 22861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:55:23.872907 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 22861
I20260929 02:55:23.873555 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:37927
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:42753
--webserver_port=33467
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:55:23 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650523, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:23.893440 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:23.873951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:43383)
0929 02:55:23.874783 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:23.874827 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:23.876540 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:23.878072 (+ 1532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:23.878079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:23.878565 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:23.879080 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:23.879098 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:23.880866 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:23.880870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:23.882032 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:23.882043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:23.882997 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:23.883004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:23.883248 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:23.883824 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:23.883849 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:23.886112 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:23.889391 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:23.889400 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:23.889415 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:23.889826 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:23.890229 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:23.890234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:23.890238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:23.890434 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:23.891099 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:23.891108 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:23.891414 (+ 306us) client_negotiation.cc:769] Sending connection context
0929 02:55:23.891650 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 02:55:23.892190 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":335,"threads_started":1}
I20260929 02:55:23.893440 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:23.873731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:43383 (local address 127.8.57.62:42753)
0929 02:55:23.874310 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:23.874324 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:23.874953 (+ 629us) server_negotiation.cc:407] Connection header received
0929 02:55:23.877122 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:23.877140 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:23.877566 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:23.877924 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:23.879298 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:23.880720 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:23.882162 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:23.882775 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:23.882945 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:23.886371 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:23.886378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:23.886392 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:23.886432 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:23.889243 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:23.889949 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:23.889955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:23.889960 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:23.890126 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:23.890535 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:23.890539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:23.890541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:23.890972 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:23.891150 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:23.891911 (+ 761us) server_negotiation.cc:299] Negotiation successful
0929 02:55:23.892296 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
I20260929 02:55:23.896715 23010 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42753
I20260929 02:55:23.896973 23010 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:23.897610 23010 heartbeater.cc:511] Master 127.8.57.62:42753 requested a full tablet report, sending...
I20260929 02:55:23.899554 22824 ts_manager.cc:194] Registered new tserver with Master: 248996eb1a4e4601867557174b158383 (127.8.57.1:37399)
I20260929 02:55:23.900995 22824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:43383
W20260929 02:55:24.181213 23014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:24.181483 23014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:24.181618 23014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:24.188669 23014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:24.188781 23014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:24.188936 23014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:55:24.198143 23014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:37927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=33467
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:24.199369 23014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:24.200599 23014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:24.212819 23020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:24.212920 23022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:24.212793 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:24.213487 23014 server_base.cc:1034] running on GCE node
I20260929 02:55:24.213999 23014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:24.214949 23014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:24.216135 23014 hybrid_clock.cc:648] HybridClock initialized: now 1790650524216088 us; error 39 us; skew 500 ppm
I20260929 02:55:24.216413 23014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:24 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:24.219267 23014 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:55:24.220557 23014 webserver.cc:460] Webserver started at http://127.8.57.2:33467/ using document root <none> and password file <none>
I20260929 02:55:24.221160 23014 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:24.221269 23014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:24.226159 23014 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:55:24.228621 23029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:24.229785 23014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:55:24.229946 23014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "e7b471895f9045d18c2c59fadee54d86"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "8c9e1f87a4c188098869efe69863117a"
server_key_iv: "4665fdecc8e22f376964daeb2df9aeeb"
server_key_version: "encryptionkey@0"
I20260929 02:55:24.230821 23014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:24.252856 23014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:24.256430 23014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:24.256661 23014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:24.257583 23014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:24.258888 23014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:24.258999 23014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:24.259101 23014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:24.259166 23014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:24.287585 23014 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:37927
I20260929 02:55:24.287640 23141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:37927 every 8 connection(s)
I20260929 02:55:24.288992 23014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:55:24.292943 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23014
I20260929 02:55:24.293529 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:45491
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:42753
--webserver_port=40885
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:55:24 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:24.308926 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.291425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:44871 (local address 127.8.57.62:42753)
0929 02:55:24.291610 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.291615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.292770 (+ 1155us) server_negotiation.cc:407] Connection header received
0929 02:55:24.294668 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.294674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.294753 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.294840 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:24.296398 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.297081 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.298610 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.299085 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.299197 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.303002 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:24.303007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:24.303011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:24.303061 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:24.305480 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:24.306402 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:24.306407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:24.306409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:24.306537 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:24.306984 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:24.306987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:24.306989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:24.307244 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:24.307348 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.308290 (+ 942us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.308556 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:55:24.311113 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.291949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:44871)
0929 02:55:24.292622 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.292662 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.294388 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.295092 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.295109 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.295613 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:24.296230 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.296250 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.297239 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.297242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.298461 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.298473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.299301 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.299310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.299523 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.300224 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:24.300255 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:24.302774 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:24.305615 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:24.305624 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:24.305641 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:24.306278 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:24.306633 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:24.306638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:24.306642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:24.306877 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:24.307355 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:24.307363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:24.307797 (+ 434us) client_negotiation.cc:769] Sending connection context
0929 02:55:24.309146 (+ 1349us) client_negotiation.cc:240] Negotiation successful
0929 02:55:24.309710 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":269,"threads_started":1}
I20260929 02:55:24.312628 23142 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42753
I20260929 02:55:24.312958 23142 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:24.313805 23142 heartbeater.cc:511] Master 127.8.57.62:42753 requested a full tablet report, sending...
I20260929 02:55:24.315392 22824 ts_manager.cc:194] Registered new tserver with Master: e7b471895f9045d18c2c59fadee54d86 (127.8.57.2:37927)
I20260929 02:55:24.316526 22824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:44871
W20260929 02:55:24.599995 23145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:24.600262 23145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:24.600358 23145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:24.607383 23145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:24.607486 23145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:24.607599 23145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:55:24.617334 23145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:45491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=40885
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:24.618518 23145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:24.619747 23145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:24.632077 23150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:24.632121 23151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:24.632194 23153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:24.632812 23145 server_base.cc:1034] running on GCE node
I20260929 02:55:24.633438 23145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:24.634351 23145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:24.635555 23145 hybrid_clock.cc:648] HybridClock initialized: now 1790650524635487 us; error 61 us; skew 500 ppm
I20260929 02:55:24.635860 23145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:24 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:24.638700 23145 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:55:24.640046 23145 webserver.cc:460] Webserver started at http://127.8.57.3:40885/ using document root <none> and password file <none>
I20260929 02:55:24.640743 23145 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:24.640895 23145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:24.646131 23145 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:55:24.648738 23160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:24.650012 23145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:55:24.650161 23145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "8a8418509cbf4362897529eda3777065"
format_stamp: "Formatted at 2026-09-29 02:55:22 on dist-test-slave-1dkt"
server_key: "acb0a433b8d8ed943f3cbbacd95b612d"
server_key_iv: "5638e9b715665ca565c05596fd690f02"
server_key_version: "encryptionkey@0"
I20260929 02:55:24.651095 23145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:24.665555 23145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:24.669492 23145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:24.669736 23145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:24.670673 23145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:24.671936 23145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:24.672029 23145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:24.672130 23145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:24.672200 23145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:24.700749 23145 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:45491
I20260929 02:55:24.700803 23272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:45491 every 8 connection(s)
I20260929 02:55:24.702090 23145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:55:24.703689 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23145
Sep 29 02:55:24 dist-test-slave-1dkt krb5kdc[22240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:24.724141 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.704637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:58411 (local address 127.8.57.62:42753)
0929 02:55:24.704815 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.704821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.706400 (+ 1579us) server_negotiation.cc:407] Connection header received
0929 02:55:24.709225 (+ 2825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.709230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.709301 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.709379 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:24.711440 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.712307 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.713799 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.714341 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.714443 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.717970 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:24.717976 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:24.717980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:24.718016 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:24.720929 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:24.721734 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:24.721740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:24.721744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:24.721889 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:24.722343 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:24.722347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:24.722349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:24.722618 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:24.722725 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.723517 (+ 792us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.723703 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260929 02:55:24.725178 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.705272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:58411)
0929 02:55:24.706214 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.706264 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.708906 (+ 2642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.709668 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.709681 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.710446 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:24.711262 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.711287 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.712441 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.712444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.713649 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.713661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.714584 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.714592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.714860 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.715597 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:24.715619 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:24.717774 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:24.721057 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:24.721067 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:24.721084 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:24.721626 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:24.721991 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:24.721995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:24.721999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:24.722236 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:24.722737 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:24.722745 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:24.723162 (+ 417us) client_negotiation.cc:769] Sending connection context
0929 02:55:24.723389 (+ 227us) client_negotiation.cc:240] Negotiation successful
0929 02:55:24.723837 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":385,"threads_started":1}
I20260929 02:55:24.725594 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.704637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.8.57.62:42753)
0929 02:55:24.712284 (+ 7647us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.712290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.712308 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:55:24.712348 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.712353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.712418 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.712506 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:24.713990 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.715026 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.716558 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.717026 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.717124 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.720103 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:24.720108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:24.720111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:24.720150 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:24.722970 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:24.723638 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:24.723643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:24.723646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:24.723780 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:24.724232 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:24.724237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:24.724241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:24.724544 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:24.724783 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.725051 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.725262 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":7526,"thread_start_us":7316,"threads_started":1}
I20260929 02:55:24.726712 23273 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42753
I20260929 02:55:24.727047 23273 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:24.727936 23273 heartbeater.cc:511] Master 127.8.57.62:42753 requested a full tablet report, sending...
I20260929 02:55:24.729538 22823 ts_manager.cc:194] Registered new tserver with Master: 8a8418509cbf4362897529eda3777065 (127.8.57.3:45491)
I20260929 02:55:24.730495 22823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:58411
I20260929 02:55:24.740845 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:55:24.746928 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.741676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.8.57.62:42753)
0929 02:55:24.741814 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.741819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.742277 (+ 458us) server_negotiation.cc:407] Connection header received
0929 02:55:24.742439 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.742443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.742576 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.742691 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:55:24.743020 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.743717 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.745089 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.745495 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.745579 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.745993 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:55:24.746499 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:55:24.746608 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.746666 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.746724 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 02:55:24.747737 22823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:55:24.754865 22823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:55:24.789477 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.765759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52386)
0929 02:55:24.766437 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.766475 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.766631 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.782138 (+ 15507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.782146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.782182 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:24.782584 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.782598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.784630 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.784635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.787036 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.787051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.788602 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.788609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.788753 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.788789 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:55:24.788868 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 02:55:24.789028 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":24,"thread_start_us":275,"threads_started":1}
I20260929 02:55:24.789844 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.766556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:37927 (local address 127.0.0.1:52230)
0929 02:55:24.780370 (+ 13814us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.780396 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.780558 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.783094 (+ 2536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.783110 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.783135 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:24.783514 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.783525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.785470 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.785478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.786893 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.786919 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.789082 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.789088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.789247 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.789280 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:55:24.789380 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:55:24.789502 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13648,"mutex_wait_us":3,"spinlock_wait_cycles":24832,"thread_start_us":14111,"threads_started":1}
I20260929 02:55:24.789907 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.767202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.8.57.2:37927)
0929 02:55:24.782328 (+ 15126us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.782338 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.782371 (+ 33us) server_negotiation.cc:407] Connection header received
0929 02:55:24.782467 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.782474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.782708 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.783690 (+ 982us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:24.783972 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.785361 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.787094 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.788871 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.788993 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.789219 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.789416 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.789570 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14958,"thread_start_us":15631,"threads_started":1}
I20260929 02:55:24.790516 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.765966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.8.57.1:37399)
0929 02:55:24.781187 (+ 15221us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.781197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.781232 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:55:24.781341 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.781351 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.781630 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.782796 (+ 1166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:24.783047 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.784507 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.787203 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.788384 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.789803 (+ 1419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.789994 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.790077 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.790226 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15020,"thread_start_us":15810,"threads_started":1}
I20260929 02:55:24.791935 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.766890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45491 (local address 127.0.0.1:53836)
0929 02:55:24.783160 (+ 16270us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.783177 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.783307 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.784291 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.784295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.784320 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:24.785168 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.785180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.787030 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.787035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.789022 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.789035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.791225 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.791230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.791348 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.791369 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 02:55:24.791436 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:55:24.791551 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16131,"spinlock_wait_cycles":896,"thread_start_us":15999,"threads_started":1}
I20260929 02:55:24.792881 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.767606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.8.57.3:45491)
0929 02:55:24.782716 (+ 15110us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.782724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.783291 (+ 567us) server_negotiation.cc:407] Connection header received
0929 02:55:24.783473 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.783480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.783699 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.783976 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:24.785351 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.786880 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.789198 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.790774 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.790887 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.791088 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:24.792415 (+ 1327us) server_negotiation.cc:299] Negotiation successful
0929 02:55:24.792562 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14947,"thread_start_us":14736,"threads_started":1}
I20260929 02:55:24.793601 23077 tablet_service.cc:1467] Processing CreateTablet for tablet 81c46d169db649bab311e21fe927e5d0 (DEFAULT_TABLE table=test-table [id=1c47b5c2252f49d4b34993cfcfb6ed5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:24.794044 22945 tablet_service.cc:1467] Processing CreateTablet for tablet 81c46d169db649bab311e21fe927e5d0 (DEFAULT_TABLE table=test-table [id=1c47b5c2252f49d4b34993cfcfb6ed5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:24.795028 23077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c46d169db649bab311e21fe927e5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:24.795174 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c46d169db649bab311e21fe927e5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:24.796247 23208 tablet_service.cc:1467] Processing CreateTablet for tablet 81c46d169db649bab311e21fe927e5d0 (DEFAULT_TABLE table=test-table [id=1c47b5c2252f49d4b34993cfcfb6ed5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:24.797503 23208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c46d169db649bab311e21fe927e5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:24.806721 23289 tablet_bootstrap.cc:492] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Bootstrap starting.
I20260929 02:55:24.809484 23290 tablet_bootstrap.cc:492] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Bootstrap starting.
I20260929 02:55:24.810207 23289 tablet_bootstrap.cc:654] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:24.811431 23291 tablet_bootstrap.cc:492] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Bootstrap starting.
I20260929 02:55:24.812376 23289 log.cc:824] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:24.813053 23290 tablet_bootstrap.cc:654] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:24.814095 23290 log.cc:824] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:24.814322 23291 tablet_bootstrap.cc:654] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:24.815411 23289 tablet_bootstrap.cc:492] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: No bootstrap required, opened a new log
I20260929 02:55:24.815418 23291 log.cc:824] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:24.815775 23289 ts_tablet_manager.cc:1397] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:55:24.816866 23290 tablet_bootstrap.cc:492] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: No bootstrap required, opened a new log
I20260929 02:55:24.817137 23290 ts_tablet_manager.cc:1397] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:55:24.822540 23291 tablet_bootstrap.cc:492] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: No bootstrap required, opened a new log
I20260929 02:55:24.822762 23291 ts_tablet_manager.cc:1397] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260929 02:55:24.826370 23289 raft_consensus.cc:348] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.826946 23289 raft_consensus.cc:374] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:24.827057 23289 raft_consensus.cc:729] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 248996eb1a4e4601867557174b158383, State: Initialized, Role: FOLLOWER
I20260929 02:55:24.827277 23290 raft_consensus.cc:348] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.827667 23290 raft_consensus.cc:374] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:24.827762 23290 raft_consensus.cc:729] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b471895f9045d18c2c59fadee54d86, State: Initialized, Role: FOLLOWER
I20260929 02:55:24.827667 23289 consensus_queue.cc:260] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.828409 23290 consensus_queue.cc:260] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.828790 23010 heartbeater.cc:503] Master 127.8.57.62:42753 was elected leader, sending a full tablet report...
I20260929 02:55:24.829428 23142 heartbeater.cc:503] Master 127.8.57.62:42753 was elected leader, sending a full tablet report...
I20260929 02:55:24.830807 23289 ts_tablet_manager.cc:1428] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260929 02:55:24.830689 23291 raft_consensus.cc:348] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.831497 23291 raft_consensus.cc:374] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:24.831625 23291 raft_consensus.cc:729] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8418509cbf4362897529eda3777065, State: Initialized, Role: FOLLOWER
I20260929 02:55:24.831959 23290 ts_tablet_manager.cc:1428] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260929 02:55:24.832240 23291 consensus_queue.cc:260] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.833428 23273 heartbeater.cc:503] Master 127.8.57.62:42753 was elected leader, sending a full tablet report...
I20260929 02:55:24.834091 23291 ts_tablet_manager.cc:1428] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
W20260929 02:55:24.872912 23011 tablet.cc:2369] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:55:24.953639 23274 tablet.cc:2369] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:24.993287 23297 raft_consensus.cc:482] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:55:24.993537 23297 raft_consensus.cc:504] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:24.995121 23297 leader_election.cc:288] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 248996eb1a4e4601867557174b158383 (127.8.57.1:37399), e7b471895f9045d18c2c59fadee54d86 (127.8.57.2:37927)
I20260929 02:55:25.000512 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.995568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:57309 (local address 127.8.57.1:37399)
0929 02:55:24.995777 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.995782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.995901 (+ 119us) server_negotiation.cc:407] Connection header received
0929 02:55:24.996099 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.996103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.996161 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.996234 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:24.996645 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.997338 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.998917 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.999674 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.999790 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:24.999902 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:25.000156 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 02:55:25.000276 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 02:55:25.000548 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.995380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.8.57.3:57309)
0929 02:55:24.995749 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.995769 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.995950 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.996280 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.996285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.996312 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:24.996498 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.996506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.997475 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.997479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.998762 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.998775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.999826 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.999832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:25.000009 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.000033 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:55:25.000115 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:55:25.000237 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":1,"spinlock_wait_cycles":36992}
I20260929 02:55:25.001075 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.995635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:33415 (local address 127.8.57.2:37927)
0929 02:55:24.995825 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:24.995830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:24.996370 (+ 540us) server_negotiation.cc:407] Connection header received
0929 02:55:24.996551 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:24.996556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:24.996625 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:24.996694 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:24.997182 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.997848 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.999446 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.000258 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.000365 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.000459 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:25.000807 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 02:55:25.000911 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:55:25.001753 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:24.995541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:37927 (local address 127.8.57.3:33415)
0929 02:55:24.996266 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:24.996280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:24.996427 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:24.996722 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:24.996727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:24.996748 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:24.997034 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.997043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:24.997985 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:24.997989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:24.999304 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:24.999316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.000438 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.000446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:25.000640 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.000672 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:55:25.001250 (+ 578us) client_negotiation.cc:240] Negotiation successful
0929 02:55:25.001366 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":21,"spinlock_wait_cycles":1408,"thread_start_us":409,"threads_started":1}
I20260929 02:55:25.001652 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c46d169db649bab311e21fe927e5d0" candidate_uuid: "8a8418509cbf4362897529eda3777065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "248996eb1a4e4601867557174b158383" is_pre_election: true
I20260929 02:55:25.002238 22965 raft_consensus.cc:2454] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a8418509cbf4362897529eda3777065 in term 0.
I20260929 02:55:25.002866 23162 leader_election.cc:302] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 248996eb1a4e4601867557174b158383, 8a8418509cbf4362897529eda3777065; no voters:
I20260929 02:55:25.002784 23097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c46d169db649bab311e21fe927e5d0" candidate_uuid: "8a8418509cbf4362897529eda3777065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7b471895f9045d18c2c59fadee54d86" is_pre_election: true
I20260929 02:55:25.003206 23297 raft_consensus.cc:2781] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:55:25.003214 23097 raft_consensus.cc:2454] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a8418509cbf4362897529eda3777065 in term 0.
I20260929 02:55:25.003317 23297 raft_consensus.cc:482] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:55:25.003399 23297 raft_consensus.cc:3037] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:25.005540 23297 raft_consensus.cc:504] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:25.006095 23297 leader_election.cc:288] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [CANDIDATE]: Term 1 election: Requested vote from peers 248996eb1a4e4601867557174b158383 (127.8.57.1:37399), e7b471895f9045d18c2c59fadee54d86 (127.8.57.2:37927)
I20260929 02:55:25.006457 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c46d169db649bab311e21fe927e5d0" candidate_uuid: "8a8418509cbf4362897529eda3777065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "248996eb1a4e4601867557174b158383"
I20260929 02:55:25.006511 23097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81c46d169db649bab311e21fe927e5d0" candidate_uuid: "8a8418509cbf4362897529eda3777065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7b471895f9045d18c2c59fadee54d86"
I20260929 02:55:25.006659 22965 raft_consensus.cc:3037] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:25.006670 23097 raft_consensus.cc:3037] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:25.008899 22965 raft_consensus.cc:2454] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a8418509cbf4362897529eda3777065 in term 1.
I20260929 02:55:25.009034 23097 raft_consensus.cc:2454] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a8418509cbf4362897529eda3777065 in term 1.
I20260929 02:55:25.009331 23162 leader_election.cc:302] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 248996eb1a4e4601867557174b158383, 8a8418509cbf4362897529eda3777065; no voters:
I20260929 02:55:25.009603 23297 raft_consensus.cc:2781] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:25.010020 23297 raft_consensus.cc:686] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 1 LEADER]: Becoming Leader. State: Replica: 8a8418509cbf4362897529eda3777065, State: Running, Role: LEADER
I20260929 02:55:25.010465 23297 consensus_queue.cc:237] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } }
I20260929 02:55:25.014822 22822 catalog_manager.cc:5495] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a8418509cbf4362897529eda3777065 (127.8.57.3). New cstate: current_term: 1 leader_uuid: "8a8418509cbf4362897529eda3777065" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a8418509cbf4362897529eda3777065" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 } health_report { overall_health: UNKNOWN } } }
W20260929 02:55:25.040768 23143 tablet.cc:2369] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:25.085067 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.080265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.8.57.3:45491)
0929 02:55:25.080471 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:25.080476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:25.080519 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:55:25.080711 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:25.080715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:25.080800 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:25.080920 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:55:25.081200 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.081904 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.083150 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.083551 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.083644 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.084081 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:55:25.084585 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:55:25.084690 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:25.084754 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 02:55:25.084815 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:55:25.092602 23097 raft_consensus.cc:1261] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Refusing update from remote peer 8a8418509cbf4362897529eda3777065: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:25.092602 22965 raft_consensus.cc:1261] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Refusing update from remote peer 8a8418509cbf4362897529eda3777065: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:25.093493 23299 consensus_queue.cc:1035] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [LEADER]: Connected to new peer: Peer: permanent_uuid: "248996eb1a4e4601867557174b158383" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:25.093839 23297 consensus_queue.cc:1035] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7b471895f9045d18c2c59fadee54d86" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:25.105450 23303 mvcc.cc:205] Tried to move back new op lower bound from 7334504550770167808 to 7334504550446583808. Current Snapshot: MvccSnapshot[applied={T|T < 7334504550770167808}]
I20260929 02:55:25.115743 22823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:25.116034 22823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:25.119119 22823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Sending DeleteTablet for 3 replicas of tablet 81c46d169db649bab311e21fe927e5d0
I20260929 02:55:25.120254 23208 tablet_service.cc:1514] Processing DeleteTablet for tablet 81c46d169db649bab311e21fe927e5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:25 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:53836
I20260929 02:55:25.120339 22945 tablet_service.cc:1514] Processing DeleteTablet for tablet 81c46d169db649bab311e21fe927e5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:25 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:52386
I20260929 02:55:25.120464 23077 tablet_service.cc:1514] Processing DeleteTablet for tablet 81c46d169db649bab311e21fe927e5d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:25 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:52230
I20260929 02:55:25.121287 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22861
I20260929 02:55:25.128325 23312 tablet_replica.cc:330] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: stopping tablet replica
I20260929 02:55:25.128836 23312 raft_consensus.cc:2229] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:25.129205 23311 tablet_replica.cc:330] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: stopping tablet replica
I20260929 02:55:25.129475 23312 raft_consensus.cc:2258] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:25.129577 23311 raft_consensus.cc:2229] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:55:25.130199 23311 raft_consensus.cc:2258] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:25.131910 23312 ts_tablet_manager.cc:1905] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:25.132529 23313 tablet_replica.cc:330] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: stopping tablet replica
I20260929 02:55:25.132915 23313 raft_consensus.cc:2229] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:25.133410 23313 raft_consensus.cc:2258] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:25.136410 23313 ts_tablet_manager.cc:1905] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:25.139322 23311 ts_tablet_manager.cc:1905] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:25.141875 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.122368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.8.57.1:37399)
0929 02:55:25.122533 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:25.122539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:25.122557 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:55:25.122616 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:25.122621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:25.122724 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:25.122827 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:25.124500 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.125712 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.128729 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.129301 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.129421 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.133486 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:25.133500 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:25.133517 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:25.133566 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:25.137426 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:25.138471 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:25.138480 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:25.138483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:25.138659 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:25.139124 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:25.139133 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:25.139136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:25.139626 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:25.141123 (+ 1497us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:25.141288 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 02:55:25.141502 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:55:25.142534 23312 ts_tablet_manager.cc:1918] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:25.142707 23312 log.cc:1196] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/81c46d169db649bab311e21fe927e5d0
I20260929 02:55:25.142699 23005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:25.144963 23313 ts_tablet_manager.cc:1918] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:25.145198 23313 log.cc:1196] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/81c46d169db649bab311e21fe927e5d0
I20260929 02:55:25.145792 23313 ts_tablet_manager.cc:1939] T 81c46d169db649bab311e21fe927e5d0 P e7b471895f9045d18c2c59fadee54d86: Deleting consensus metadata
I20260929 02:55:25.147728 22808 catalog_manager.cc:4841] TS e7b471895f9045d18c2c59fadee54d86 (127.8.57.2:37927): tablet 81c46d169db649bab311e21fe927e5d0 (table test-table [id=1c47b5c2252f49d4b34993cfcfb6ed5b]) successfully deleted
I20260929 02:55:25.148326 23311 ts_tablet_manager.cc:1918] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:25.148515 23311 log.cc:1196] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/81c46d169db649bab311e21fe927e5d0
I20260929 02:55:25.149027 23311 ts_tablet_manager.cc:1939] T 81c46d169db649bab311e21fe927e5d0 P 8a8418509cbf4362897529eda3777065: Deleting consensus metadata
I20260929 02:55:25.150745 22809 catalog_manager.cc:4841] TS 8a8418509cbf4362897529eda3777065 (127.8.57.3:45491): tablet 81c46d169db649bab311e21fe927e5d0 (table test-table [id=1c47b5c2252f49d4b34993cfcfb6ed5b]) successfully deleted
I20260929 02:55:25.256831 23312 ts_tablet_manager.cc:1939] T 81c46d169db649bab311e21fe927e5d0 P 248996eb1a4e4601867557174b158383: Deleting consensus metadata
I20260929 02:55:25.257107 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22861
W20260929 02:55:25.271271 22809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.57.1:37399 (error 108)
I20260929 02:55:25.271556 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23014
I20260929 02:55:25.274904 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.274408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52396)
0929 02:55:25.274601 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.274730 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260929 02:55:25.275192 22809 catalog_manager.cc:4584] TS 248996eb1a4e4601867557174b158383 (127.8.57.1:37399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81c46d169db649bab311e21fe927e5d0: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
I20260929 02:55:25.284871 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.272391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.8.57.2:37927)
0929 02:55:25.272563 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:25.272569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:25.272588 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:55:25.272753 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:25.272759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:25.272833 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:25.272923 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:25.274234 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.275122 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.276554 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.276945 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.277057 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.279412 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:25.279426 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:25.279440 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:25.279472 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:25.282174 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:25.282847 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:25.282855 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:25.282857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:25.282980 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:25.283312 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:25.283316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:25.283318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:25.283648 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:25.283935 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:25.284342 (+ 407us) server_negotiation.cc:299] Negotiation successful
0929 02:55:25.284547 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:55:25.285522 23137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:25.328796 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.328281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52410)
0929 02:55:25.328496 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.328627 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 02:55:25.329838 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.329473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52420)
0929 02:55:25.329612 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.329706 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 02:55:25.373018 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23014
I20260929 02:55:25.386687 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23145
I20260929 02:55:25.390499 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.389988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52436)
0929 02:55:25.390190 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.390290 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 02:55:25.391611 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.391218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52448)
0929 02:55:25.391374 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.391469 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 02:55:25.401168 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.387763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.8.57.3:45491)
0929 02:55:25.387972 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:25.387977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:25.388049 (+ 72us) server_negotiation.cc:407] Connection header received
0929 02:55:25.388253 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:25.388257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:25.388319 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:25.388412 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:25.389720 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.390759 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.392402 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:25.392850 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:25.392960 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:25.395389 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:25.395400 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:25.395417 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:25.395453 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:25.398373 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:25.399041 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:25.399048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:25.399051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:25.399221 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:25.399581 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:25.399585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:25.399587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:25.399972 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:25.400249 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:25.400559 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 02:55:25.400760 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:55:25.401877 23268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:25.472213 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.471707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52464)
0929 02:55:25.471939 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.472026 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 02:55:25.473273 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.472896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52478)
0929 02:55:25.473083 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.473153 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260929 02:55:25.520120 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23145
I20260929 02:55:25.534970 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 22791
I20260929 02:55:25.536093 22854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:25.647167 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:25.646543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:37399 (local address 127.0.0.1:52488)
0929 02:55:25.646753 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:25.646857 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 02:55:25.647334 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 22791
I20260929 02:55:25.664060 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42753
--webserver_interface=127.8.57.62
--webserver_port=39067
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:25.972594 23319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:25.972892 23319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:25.973029 23319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:25.980199 23319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:25.980319 23319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:25.980396 23319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:25.980468 23319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:25.980528 23319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:25.990272 23319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=39067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:25.991554 23319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:25.992852 23319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:26.004823 23325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:26.004892 23324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:26.004856 23327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:26.005614 23319 server_base.cc:1034] running on GCE node
I20260929 02:55:26.006438 23319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:26.008301 23319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:26.009558 23319 hybrid_clock.cc:648] HybridClock initialized: now 1790650526009487 us; error 62 us; skew 500 ppm
I20260929 02:55:26.009966 23319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:26 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:26.013240 23319 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:26.014861 23319 webserver.cc:460] Webserver started at http://127.8.57.62:39067/ using document root <none> and password file <none>
I20260929 02:55:26.015558 23319 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:26.015700 23319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:26.021415 23319 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:55:26.024296 23334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.025569 23319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:55:26.025718 23319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "1c6a62e7544f48a792e0cde7dbb0dd17"
format_stamp: "Formatted at 2026-09-29 02:55:20 on dist-test-slave-1dkt"
server_key: "642f9003bdf9ec4ca8feb3e1a05dc4ec"
server_key_iv: "7c76ed8e0f9dcfddaa5b297815fc4ea6"
server_key_version: "encryptionkey@0"
I20260929 02:55:26.026648 23319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:26.069753 23319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:26.074159 23319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:26.074434 23319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:26.091740 23319 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42753
I20260929 02:55:26.091766 23385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42753 every 8 connection(s)
I20260929 02:55:26.093272 23319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:26.100391 23386 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:55:26.103273 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23319
I20260929 02:55:26.103729 23386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap starting.
I20260929 02:55:26.104074 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:37399
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:42753
--webserver_port=46595
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:26.123274 23386 log.cc:824] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:26.139047 23386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:55:26.139518 23386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17: Bootstrap complete.
I20260929 02:55:26.148298 23386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:26.149144 23386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c6a62e7544f48a792e0cde7dbb0dd17, State: Initialized, Role: FOLLOWER
I20260929 02:55:26.149680 23386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:26.149888 23386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:26.149994 23386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:55:26.150138 23386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 2 FOLLOWER]: Advancing to term 3
I20260929 02:55:26.153738 23386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:26.154179 23386 leader_election.cc:302] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c6a62e7544f48a792e0cde7dbb0dd17; no voters:
I20260929 02:55:26.154726 23386 leader_election.cc:288] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 02:55:26.154910 23390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 02:55:26.155777 23390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [term 3 LEADER]: Becoming Leader. State: Replica: 1c6a62e7544f48a792e0cde7dbb0dd17, State: Running, Role: LEADER
I20260929 02:55:26.156556 23386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:26.156432 23390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } }
I20260929 02:55:26.159149 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c6a62e7544f48a792e0cde7dbb0dd17. Latest consensus state: current_term: 3 leader_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } } }
I20260929 02:55:26.159494 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:26.160080 23391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c6a62e7544f48a792e0cde7dbb0dd17" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42753 } } }
I20260929 02:55:26.160336 23391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6a62e7544f48a792e0cde7dbb0dd17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:26.176836 23398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:26.185381 23398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:26.186853 23398 catalog_manager.cc:1259] Loaded cluster ID: 9e2fe184c00545b2991acffe828834d2
I20260929 02:55:26.187024 23398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 02:55:26.189028 23398 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2d9fec79ad google::LogMessage::Fail() at ??:0
@ 0x7f2d9feca88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2d9fec74de google::LogMessage::Flush() at ??:0
@ 0x7f2d9fec8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2db58fa512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2db58f7976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2db597f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2db597f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2da2b13345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2da2b1691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2da2b16799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2da2af1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2da5dcb6db start_thread at ??:0
@ 0x7f2d9ed6161f clone at ??:0
W20260929 02:55:26.486088 23388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:26.486342 23388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:26.486443 23388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:26.493422 23388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:26.493523 23388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:26.493641 23388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:55:26.503337 23388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:37399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=46595
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:26.504631 23388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:26.505929 23388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:26.518296 23428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:26.518333 23429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:26.518371 23431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:26.518918 23388 server_base.cc:1034] running on GCE node
I20260929 02:55:26.519493 23388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:26.520442 23388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:26.521651 23388 hybrid_clock.cc:648] HybridClock initialized: now 1790650526521598 us; error 40 us; skew 500 ppm
I20260929 02:55:26.521948 23388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:26 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:26.524816 23388 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:55:26.526118 23388 webserver.cc:460] Webserver started at http://127.8.57.1:46595/ using document root <none> and password file <none>
I20260929 02:55:26.526746 23388 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:26.526847 23388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:26.531906 23388 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:26.534350 23438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.535570 23388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:55:26.535723 23388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "248996eb1a4e4601867557174b158383"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "2ea48f476d2bb9a46ea0501abcb373c1"
server_key_iv: "056bc46ca97f44b087d7e61d60fafde2"
server_key_version: "encryptionkey@0"
I20260929 02:55:26.536676 23388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:26.552380 23388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:26.556005 23388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:26.556234 23388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:26.557082 23388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:26.558290 23388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:26.558380 23388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.558480 23388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:26.558554 23388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.587445 23388 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:37399
I20260929 02:55:26.587498 23550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:37399 every 8 connection(s)
I20260929 02:55:26.588846 23388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:55:26.593299 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:26.591544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:41117)
0929 02:55:26.592288 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:26.592469 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":315,"threads_started":1}
W20260929 02:55:26.593919 23551 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:42753 (0 consecutive failures): Network error: Failed to ping master at 127.8.57.62:42753: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
I20260929 02:55:26.594516 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23388
I20260929 02:55:26.594782 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:26.594379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:59575)
0929 02:55:26.594559 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:26.594636 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 02:55:26.595173 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:37927
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:42753
--webserver_port=33467
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:26.595538 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:26.595240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:55685)
0929 02:55:26.595366 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:26.595437 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 02:55:26.595772 23551 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:55:26.899276 23554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:26.899562 23554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:26.899699 23554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:26.906977 23554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:26.907097 23554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:26.907245 23554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:55:26.916998 23554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:37927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=33467
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:26.918217 23554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:26.919459 23554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:26.931571 23560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:26.931631 23562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:26.931640 23559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:26.932610 23554 server_base.cc:1034] running on GCE node
I20260929 02:55:26.933156 23554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:26.934047 23554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:26.935236 23554 hybrid_clock.cc:648] HybridClock initialized: now 1790650526935189 us; error 38 us; skew 500 ppm
I20260929 02:55:26.935511 23554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:26 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:26.938369 23554 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:55:26.939728 23554 webserver.cc:460] Webserver started at http://127.8.57.2:33467/ using document root <none> and password file <none>
I20260929 02:55:26.940438 23554 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:26.940550 23554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:26.945739 23554 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:55:26.948444 23569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.949752 23554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:26.949903 23554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "e7b471895f9045d18c2c59fadee54d86"
format_stamp: "Formatted at 2026-09-29 02:55:21 on dist-test-slave-1dkt"
server_key: "8c9e1f87a4c188098869efe69863117a"
server_key_iv: "4665fdecc8e22f376964daeb2df9aeeb"
server_key_version: "encryptionkey@0"
I20260929 02:55:26.950839 23554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:26.965221 23554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:26.968745 23554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:26.968995 23554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:26.969830 23554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:26.971055 23554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:26.971134 23554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.971210 23554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:26.971262 23554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:26.999555 23554 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:37927
I20260929 02:55:26.999616 23681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:37927 every 8 connection(s)
I20260929 02:55:27.000900 23554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:55:27.002831 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23554
I20260929 02:55:27.003367 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:45491
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:42753
--webserver_port=40885
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:55:27.005838 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.003822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:60069)
0929 02:55:27.004718 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.004945 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":312,"threads_started":1}
W20260929 02:55:27.006440 23682 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:42753 (0 consecutive failures): Network error: Failed to ping master at 127.8.57.62:42753: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
I20260929 02:55:27.007349 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.006970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:42009)
0929 02:55:27.007170 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.007238 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 02:55:27.008160 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.007842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:59337)
0929 02:55:27.007971 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.008043 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 02:55:27.008378 23682 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:55:27.312636 23685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:27.312899 23685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:27.313004 23685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:27.320077 23685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:27.320173 23685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:27.320286 23685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:55:27.329756 23685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:45491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=40885
--tserver_master_addrs=127.8.57.62:42753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:27.330894 23685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:27.332142 23685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:27.344213 23691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:27.344275 23690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:27.344251 23693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:27.345351 23685 server_base.cc:1034] running on GCE node
I20260929 02:55:27.345881 23685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:27.346706 23685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:27.347918 23685 hybrid_clock.cc:648] HybridClock initialized: now 1790650527347838 us; error 49 us; skew 500 ppm
I20260929 02:55:27.348219 23685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:27 dist-test-slave-1dkt krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650527, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:27.351109 23685 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:55:27.352434 23685 webserver.cc:460] Webserver started at http://127.8.57.3:40885/ using document root <none> and password file <none>
I20260929 02:55:27.353060 23685 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:27.353152 23685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:27.358168 23685 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:55:27.360662 23700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:27.361905 23685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:55:27.362030 23685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "8a8418509cbf4362897529eda3777065"
format_stamp: "Formatted at 2026-09-29 02:55:22 on dist-test-slave-1dkt"
server_key: "acb0a433b8d8ed943f3cbbacd95b612d"
server_key_iv: "5638e9b715665ca565c05596fd690f02"
server_key_version: "encryptionkey@0"
I20260929 02:55:27.362931 23685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:27.382875 23685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:27.386981 23685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:27.387213 23685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:27.388162 23685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:27.389405 23685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:27.389503 23685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:27.389581 23685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:27.389642 23685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:27.419013 23685 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:45491
I20260929 02:55:27.419080 23812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:45491 every 8 connection(s)
I20260929 02:55:27.420434 23685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:55:27.424958 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.423222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:57495)
0929 02:55:27.423917 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.424139 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":263,"threads_started":1}
I20260929 02:55:27.425416 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23685
W20260929 02:55:27.425568 23813 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:42753 (0 consecutive failures): Network error: Failed to ping master at 127.8.57.62:42753: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
I20260929 02:55:27.426437 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.426048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:46083)
0929 02:55:27.426215 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.426295 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 02:55:27.427421 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.427045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:40181)
0929 02:55:27.427242 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.427315 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260929 02:55:27.427646 23813 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:55:27.596951 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:27.596260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:60987)
0929 02:55:27.596656 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:27.596738 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":184,"threads_started":1}
I20260929 02:55:28.009555 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:28.008826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:43647)
0929 02:55:28.009254 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:28.009337 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":194,"threads_started":1}
I20260929 02:55:28.428817 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:28.428125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:48819)
0929 02:55:28.428528 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:28.428606 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":187,"threads_started":1}
I20260929 02:55:28.598659 23820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:28.597889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:59153)
0929 02:55:28.598382 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:28.598483 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":239,"threads_started":1}
I20260929 02:55:29.011247 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:29.010468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:60529)
0929 02:55:29.010974 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:29.011069 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":252,"threads_started":1}
I20260929 02:55:29.430397 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:29.429712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:37335)
0929 02:55:29.430148 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:29.430234 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":210,"threads_started":1}
I20260929 02:55:29.600219 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:29.599480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:54051)
0929 02:55:29.599943 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:29.600028 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":217,"threads_started":1}
I20260929 02:55:30.012740 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:30.012064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:54223)
0929 02:55:30.012489 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:30.012562 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":214,"threads_started":1}
I20260929 02:55:30.431769 23825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:30.431162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:60907)
0929 02:55:30.431533 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:30.431610 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":194,"threads_started":1}
I20260929 02:55:30.601764 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:30.601092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:44045)
0929 02:55:30.601496 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:30.601574 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":201,"threads_started":1}
I20260929 02:55:31.014297 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:31.013535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:57977)
0929 02:55:31.014027 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:31.014106 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":252,"threads_started":1}
I20260929 02:55:31.433287 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:31.432605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:59331)
0929 02:55:31.433008 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:31.433085 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":197,"threads_started":1}
I20260929 02:55:31.603236 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:31.602539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:50393)
0929 02:55:31.603002 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:31.603075 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":250,"threads_started":1}
I20260929 02:55:32.015760 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:32.015087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:45121)
0929 02:55:32.015513 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:32.015592 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":211,"threads_started":1}
I20260929 02:55:32.434867 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:32.434185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:44739)
0929 02:55:32.434623 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:32.434703 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":218,"threads_started":1}
I20260929 02:55:32.604859 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:32.604152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:44803)
0929 02:55:32.604593 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:32.604697 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":230,"threads_started":1}
I20260929 02:55:33.017354 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:33.016664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:58703)
0929 02:55:33.017099 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:33.017187 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":220,"threads_started":1}
I20260929 02:55:33.436419 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:33.435698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:44131)
0929 02:55:33.436133 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:33.436240 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260929 02:55:33.606587 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:33.605698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:51567)
0929 02:55:33.606220 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:33.606319 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":223,"threads_started":1}
I20260929 02:55:34.019038 23836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:34.018239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:36205)
0929 02:55:34.018737 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:34.018816 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":226,"threads_started":1}
I20260929 02:55:34.437964 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:34.437199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:41291)
0929 02:55:34.437673 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:34.437757 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":211,"threads_started":1}
I20260929 02:55:34.608199 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:34.607508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:37499)
0929 02:55:34.607960 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:34.608036 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":206,"threads_started":1}
I20260929 02:55:35.020602 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:35.019868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:56921)
0929 02:55:35.020358 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:35.020438 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":218,"threads_started":1}
I20260929 02:55:35.439489 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:35.438789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:56621)
0929 02:55:35.439220 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:35.439297 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260929 02:55:35.609658 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:35.608949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:40467)
0929 02:55:35.609412 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:35.609487 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":250,"threads_started":1}
I20260929 02:55:36.022253 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:36.021458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:52951)
0929 02:55:36.021972 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:36.022058 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":262,"threads_started":1}
I20260929 02:55:36.441179 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:36.440389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:57491)
0929 02:55:36.440876 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:36.440958 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":235,"threads_started":1}
I20260929 02:55:36.611240 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:36.610511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:47119)
0929 02:55:36.610994 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:36.611076 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":249,"threads_started":1}
I20260929 02:55:37.023780 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:37.023123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:42891)
0929 02:55:37.023537 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:37.023613 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":220,"threads_started":1}
I20260929 02:55:37.442752 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:37.442023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:35735)
0929 02:55:37.442498 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:37.442575 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":254,"threads_started":1}
I20260929 02:55:37.612774 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:37.612084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:37315)
0929 02:55:37.612503 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:37.612600 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":204,"threads_started":1}
I20260929 02:55:38.025437 23848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:38.024702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:48163)
0929 02:55:38.025168 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:38.025259 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
I20260929 02:55:38.444283 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:38.443604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:52937)
0929 02:55:38.444019 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:38.444105 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":193,"threads_started":1}
I20260929 02:55:38.614250 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:38.613619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:34737)
0929 02:55:38.614020 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:38.614096 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":205,"threads_started":1}
I20260929 02:55:39.027060 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:39.026368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:42237)
0929 02:55:39.026807 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:39.026888 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":214,"threads_started":1}
I20260929 02:55:39.445741 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:39.445106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:47487)
0929 02:55:39.445508 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:39.445592 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":207,"threads_started":1}
I20260929 02:55:39.615969 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:39.615184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:44107)
0929 02:55:39.615657 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:39.615758 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":234,"threads_started":1}
I20260929 02:55:40.028683 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:40.027942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:56413)
0929 02:55:40.028410 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:40.028491 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":232,"threads_started":1}
I20260929 02:55:40.447327 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:40.446587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:55921)
0929 02:55:40.447034 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:40.447135 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":221,"threads_started":1}
I20260929 02:55:40.617655 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:40.616893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:58999)
0929 02:55:40.617364 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:40.617452 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260929 02:55:41.030206 23857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:41.029534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:40585)
0929 02:55:41.029954 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:41.030029 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":221,"threads_started":1}
I20260929 02:55:41.448809 23858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:41.448144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:44117)
0929 02:55:41.448547 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:41.448624 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":199,"threads_started":1}
I20260929 02:55:41.619603 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:41.618852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.1:58599)
0929 02:55:41.619308 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:41.619400 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260929 02:55:42.032043 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:42.031264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.2:60009)
0929 02:55:42.031720 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:42.031812 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":236,"threads_started":1}
I20260929 02:55:42.435706 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23388
I20260929 02:55:42.450580 23862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:42.449853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42753 (local address 127.8.57.3:53521)
0929 02:55:42.450305 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:42.450388 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:42753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":244,"threads_started":1}
I20260929 02:55:42.457340 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:42.436818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53602 (local address 127.8.57.1:37399)
0929 02:55:42.437352 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:42.437370 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:42.437416 (+ 46us) server_negotiation.cc:407] Connection header received
0929 02:55:42.439151 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:42.439160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:42.439612 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:42.439937 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:42.441442 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:42.442664 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:42.443945 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:42.444602 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:42.444749 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:42.451455 (+ 6706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:42.451463 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:42.451480 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:42.451521 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:42.454167 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:42.454851 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:42.454858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:42.454864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:42.455036 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:42.455395 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:42.455398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:42.455400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:42.455947 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:42.456242 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:42.456678 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 02:55:42.457064 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":214,"threads_started":1}
I20260929 02:55:42.458634 23546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:42.540544 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23388
I20260929 02:55:42.553777 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23554
I20260929 02:55:42.570220 23864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:42.554657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.8.57.2:37927)
0929 02:55:42.555247 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:42.555260 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:42.555296 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:55:42.556803 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:42.556812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:42.557222 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:42.557503 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:42.558861 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:42.560122 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:42.561333 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:42.561952 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:42.562073 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:42.564447 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:42.564454 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:42.564468 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:42.564502 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:42.567326 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:42.568125 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:42.568132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:42.568138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:42.568304 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:42.568591 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:42.568594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:42.568596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:42.569069 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:42.569310 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:42.569576 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 02:55:42.569934 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":290,"threads_started":1}
I20260929 02:55:42.571516 23677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:42.653362 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23554
I20260929 02:55:42.667375 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23685
I20260929 02:55:42.684026 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:42.668387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.8.57.3:45491)
0929 02:55:42.669003 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:42.669017 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:42.669055 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:55:42.670614 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:42.670623 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:42.671030 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:42.671313 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:42.672722 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:42.673923 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:42.675093 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:42.675751 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:42.676007 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:42.678321 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:42.678329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:42.678342 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:42.678384 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:42.681170 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:42.681834 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:42.681840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:42.681846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:42.681999 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:42.682282 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:42.682285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:42.682287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:42.682760 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:42.682977 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:42.683343 (+ 366us) server_negotiation.cc:299] Negotiation successful
0929 02:55:42.683698 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":317,"threads_started":1}
I20260929 02:55:42.685300 23808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:42.767150 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23685
2026-09-29T02:55:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23798 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:55:42 dist-test-slave-1dkt krb5kdc[23870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:55:42 dist-test-slave-1dkt krb5kdc[23870](info): set up 2 sockets
Sep 29 02:55:42 dist-test-slave-1dkt krb5kdc[23870](info): commencing operation
krb5kdc: starting...
W20260929 02:55:43.772210 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:55:43 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:55:43.981666 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41191
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:44.300169 23883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:44.300478 23883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:44.300616 23883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:44.307847 23883 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 02:55:44.307967 23883 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 02:55:44.308032 23883 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 02:55:44.308101 23883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:55:44.308172 23883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:44.308264 23883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:44.308323 23883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:55:44.308393 23883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:55:44.318100 23883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:44.319314 23883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:44.320676 23883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:44.331990 23891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:44.332039 23889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:44.332067 23888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:44.332841 23883 server_base.cc:1034] running on GCE node
I20260929 02:55:44.333577 23883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:44.334844 23883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:44.336054 23883 hybrid_clock.cc:648] HybridClock initialized: now 1790650544335991 us; error 35 us; skew 500 ppm
I20260929 02:55:44.336400 23883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:44 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:44.339344 23883 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:55:44.340689 23883 webserver.cc:460] Webserver started at http://127.8.57.62:34399/ using document root <none> and password file <none>
I20260929 02:55:44.341331 23883 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:44.341411 23883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:44.341661 23883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:44.344416 23883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "4e5ea13a287842959dfd3d14eb0b5fab"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "89ae472867c2b3904416bbce2c983e1f"
server_key_iv: "84af357e4d476f5e1b580c51044eaaa2"
server_key_version: "encryptionkey@0"
I20260929 02:55:44.345122 23883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "4e5ea13a287842959dfd3d14eb0b5fab"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "89ae472867c2b3904416bbce2c983e1f"
server_key_iv: "84af357e4d476f5e1b580c51044eaaa2"
server_key_version: "encryptionkey@0"
I20260929 02:55:44.349830 23883 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:55:44.353046 23898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:44.354331 23883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:55:44.354496 23883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "4e5ea13a287842959dfd3d14eb0b5fab"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "89ae472867c2b3904416bbce2c983e1f"
server_key_iv: "84af357e4d476f5e1b580c51044eaaa2"
server_key_version: "encryptionkey@0"
I20260929 02:55:44.354679 23883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:44.367321 23883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:44.371117 23883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:44.371358 23883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:44.388422 23883 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41191
I20260929 02:55:44.388470 23949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41191 every 8 connection(s)
I20260929 02:55:44.389701 23883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:55:44.391848 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23883
I20260929 02:55:44.392061 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:55:44.392696 8420 external_mini_cluster.cc:1428] Setting key a3846d024de899ba6e3c91e406b21435
I20260929 02:55:44.393328 23950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:55:44 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:44.406585 23950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap starting.
I20260929 02:55:44.410292 23950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:44.411489 23950 log.cc:824] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:44.414027 23950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: No bootstrap required, opened a new log
I20260929 02:55:44.414851 23953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:44.394523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.8.57.62:41191)
0929 02:55:44.395318 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:44.395331 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:44.395378 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:55:44.396978 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:44.397003 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:44.397478 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:44.397896 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:44.399152 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:44.401144 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:44.402475 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:44.403441 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:44.403615 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:44.407395 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:44.407405 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:44.407421 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:44.407463 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:44.411082 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:44.411731 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:44.411737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:44.411743 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:44.411954 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:44.412327 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:44.412332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:44.412339 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:44.412841 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:44.413093 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:44.413524 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 02:55:44.413898 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":348,"threads_started":1}
I20260929 02:55:44.421257 23950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:55:44.421581 23950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:44.421667 23950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5ea13a287842959dfd3d14eb0b5fab, State: Initialized, Role: FOLLOWER
I20260929 02:55:44.422135 23950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:55:44.422338 23950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:55:44.422452 23950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:55:44.422590 23950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:44.424861 23950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:55:44.425297 23950 leader_election.cc:302] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e5ea13a287842959dfd3d14eb0b5fab; no voters:
I20260929 02:55:44.425828 23950 leader_election.cc:288] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:55:44.425930 23955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:44.426453 23955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 LEADER]: Becoming Leader. State: Replica: 4e5ea13a287842959dfd3d14eb0b5fab, State: Running, Role: LEADER
I20260929 02:55:44.426975 23955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:55:44.428596 23955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e5ea13a287842959dfd3d14eb0b5fab. Latest consensus state: current_term: 1 leader_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } } }
I20260929 02:55:44.428845 23955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:44.429316 23957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } } }
I20260929 02:55:44.429682 23957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: This master's current role is: LEADER
I20260929 02:55:44.430953 23950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:55:44.431819 23959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:55:44.436539 23959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:55:44.444833 23959 catalog_manager.cc:1347] Generated new cluster ID: d841d6af8e6d477d8c922e9a8c15087c
I20260929 02:55:44.444934 23959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:55:44.455248 23959 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:55:44.456548 23959 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:55:44.464592 23959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Generated new TSK 0
I20260929 02:55:44.465492 23959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:55:44.519680 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41191
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:44.832432 23976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:44.832729 23976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:44.832849 23976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:44.840636 23976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:44.840741 23976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:44.840900 23976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:55:44.850521 23976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:44.851776 23976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:44.853204 23976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:44.865773 23984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:44.865792 23982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:44.865967 23981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:44.866539 23976 server_base.cc:1034] running on GCE node
I20260929 02:55:44.867177 23976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:44.868094 23976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:44.869300 23976 hybrid_clock.cc:648] HybridClock initialized: now 1790650544869249 us; error 45 us; skew 500 ppm
I20260929 02:55:44.869596 23976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:44 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:44.872689 23976 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:55:44.874089 23976 webserver.cc:460] Webserver started at http://127.8.57.1:44789/ using document root <none> and password file <none>
I20260929 02:55:44.874764 23976 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:44.874878 23976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:44.875171 23976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:44.878046 23976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "c8188632b7354b4f8c75ee098bb17b6e"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "48764635a77c90f580dae7686df3305c"
server_key_iv: "0b3020141bf6ab7ede22cae5bd42713b"
server_key_version: "encryptionkey@0"
I20260929 02:55:44.878815 23976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "c8188632b7354b4f8c75ee098bb17b6e"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "48764635a77c90f580dae7686df3305c"
server_key_iv: "0b3020141bf6ab7ede22cae5bd42713b"
server_key_version: "encryptionkey@0"
I20260929 02:55:44.883800 23976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260929 02:55:44.887207 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:44.888717 23976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:55:44.888868 23976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "c8188632b7354b4f8c75ee098bb17b6e"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "48764635a77c90f580dae7686df3305c"
server_key_iv: "0b3020141bf6ab7ede22cae5bd42713b"
server_key_version: "encryptionkey@0"
I20260929 02:55:44.889061 23976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:44.909776 23976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:44.913719 23976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:44.913967 23976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:44.914891 23976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:44.916168 23976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:44.916267 23976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:44.916385 23976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:44.916442 23976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:44.948628 23976 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:46703
I20260929 02:55:44.948701 24103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:46703 every 8 connection(s)
I20260929 02:55:44.949855 23976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:55:44.960201 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 23976
I20260929 02:55:44.960397 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:55:44.961001 8420 external_mini_cluster.cc:1428] Setting key 625c6c1f8d56badfaaf0cd4247d91a76
Sep 29 02:55:44 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:44.969161 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:44.952333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:56015 (local address 127.8.57.62:41191)
0929 02:55:44.952782 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:44.952791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:44.953794 (+ 1003us) server_negotiation.cc:407] Connection header received
0929 02:55:44.955583 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:44.955588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:44.955666 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:44.955762 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:44.957227 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:44.957989 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:44.959571 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:44.960038 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:44.960152 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:44.963524 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:44.963530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:44.963534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:44.963569 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:44.966286 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:44.966971 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:44.966975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:44.966978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:44.967108 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:44.967500 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:44.967503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:44.967505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:44.967781 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:44.967936 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:44.968585 (+ 649us) server_negotiation.cc:299] Negotiation successful
0929 02:55:44.968829 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":213,"threads_started":1}
I20260929 02:55:44.970060 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:44.952896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:56015)
0929 02:55:44.953632 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:44.953669 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:44.955323 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:44.956006 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:44.956014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:44.956505 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:44.957054 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:44.957076 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:44.958134 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:44.958138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:44.959428 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:44.959440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:44.960269 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:44.960276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:44.960505 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:44.961151 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:44.961174 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:44.963290 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:44.966420 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:44.966428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:44.966453 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:44.966854 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:44.967210 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:44.967215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:44.967219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:44.967396 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:44.967923 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:44.967931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:44.968252 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 02:55:44.968483 (+ 231us) client_negotiation.cc:240] Negotiation successful
0929 02:55:44.968971 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":293,"threads_started":1}
I20260929 02:55:44.971510 24104 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41191
I20260929 02:55:44.971833 24104 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:44.972653 24104 heartbeater.cc:511] Master 127.8.57.62:41191 requested a full tablet report, sending...
I20260929 02:55:44.974920 23915 ts_manager.cc:194] Registered new tserver with Master: c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1:46703)
I20260929 02:55:44.976843 23915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:56015
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:55:45.015367 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41191
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:45.339433 24110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:45.339668 24110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:45.339785 24110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:45.346692 24110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:45.346782 24110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:45.346920 24110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:55:45.356045 24110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:45.357193 24110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:45.358438 24110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:45.370254 24118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:45.370297 24115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:45.370302 24116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:45.371280 24110 server_base.cc:1034] running on GCE node
I20260929 02:55:45.371855 24110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:45.372723 24110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:45.373932 24110 hybrid_clock.cc:648] HybridClock initialized: now 1790650545373871 us; error 48 us; skew 500 ppm
I20260929 02:55:45.374217 24110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:45 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650545, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:45.377280 24110 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:55:45.378540 24110 webserver.cc:460] Webserver started at http://127.8.57.2:33705/ using document root <none> and password file <none>
I20260929 02:55:45.379161 24110 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:45.379271 24110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:45.379540 24110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:45.382256 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "c4065c86f928a53e917bdb87dd56e436"
server_key_iv: "500a7f1036640e8ace4f6716e47e04e6"
server_key_version: "encryptionkey@0"
I20260929 02:55:45.382964 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "c4065c86f928a53e917bdb87dd56e436"
server_key_iv: "500a7f1036640e8ace4f6716e47e04e6"
server_key_version: "encryptionkey@0"
I20260929 02:55:45.387837 24110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260929 02:55:45.392010 24125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:45.393587 24110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:55:45.393788 24110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "c4065c86f928a53e917bdb87dd56e436"
server_key_iv: "500a7f1036640e8ace4f6716e47e04e6"
server_key_version: "encryptionkey@0"
I20260929 02:55:45.393989 24110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:45.417245 24110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:45.420852 24110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:45.421078 24110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:45.421926 24110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:45.423096 24110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:45.423167 24110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:45.423255 24110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:45.423295 24110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:45.452032 24110 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:43679
I20260929 02:55:45.452060 24237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:43679 every 8 connection(s)
I20260929 02:55:45.453233 24110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:55:45.456321 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24110
I20260929 02:55:45.456486 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:55:45.457018 8420 external_mini_cluster.cc:1428] Setting key ee2c76acd3028f14bb51f1adf77cce1c
Sep 29 02:55:45 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650545, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:45.473500 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.455574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:35959 (local address 127.8.57.62:41191)
0929 02:55:45.455807 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:45.455813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:45.457094 (+ 1281us) server_negotiation.cc:407] Connection header received
0929 02:55:45.459220 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:45.459224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:45.459287 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:45.459365 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:45.461470 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.462165 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.464156 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.464542 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.464679 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:45.468272 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:45.468277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:45.468280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:45.468311 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:45.470580 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:45.471386 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:45.471390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:45.471392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:45.471525 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:45.471958 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:45.471961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:45.471963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:45.472221 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:45.472329 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:45.473086 (+ 757us) server_negotiation.cc:299] Negotiation successful
0929 02:55:45.473262 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260929 02:55:45.474677 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.456093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:35959)
0929 02:55:45.456917 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:45.456976 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:45.458959 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:45.459622 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:45.459634 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:45.460318 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:45.461065 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.461088 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.462297 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.462302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.464034 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.464049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.464743 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.464749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.464992 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:45.465662 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:45.465694 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:45.468061 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:45.470706 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:45.470713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:45.470728 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:45.471273 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:45.471614 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:45.471619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:45.471622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:45.471829 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:45.472330 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:45.472337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:45.472726 (+ 389us) client_negotiation.cc:769] Sending connection context
0929 02:55:45.472985 (+ 259us) client_negotiation.cc:240] Negotiation successful
0929 02:55:45.473469 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":295,"threads_started":1}
I20260929 02:55:45.476027 24238 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41191
I20260929 02:55:45.476315 24238 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:45.476966 24238 heartbeater.cc:511] Master 127.8.57.62:41191 requested a full tablet report, sending...
I20260929 02:55:45.478389 23915 ts_manager.cc:194] Registered new tserver with Master: 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679)
I20260929 02:55:45.479157 23915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:35959
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:55:45.507740 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41191
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:55:45.815774 24243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:55:45.816075 24243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:55:45.816227 24243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:55:45.823094 24243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:55:45.823208 24243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:55:45.823371 24243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:55:45.832667 24243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:55:45.833853 24243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:55:45.835080 24243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:55:45.847302 24248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:45.847323 24251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:55:45.847235 24249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:55:45.848362 24243 server_base.cc:1034] running on GCE node
I20260929 02:55:45.848997 24243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:55:45.849915 24243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:55:45.851106 24243 hybrid_clock.cc:648] HybridClock initialized: now 1790650545851047 us; error 49 us; skew 500 ppm
I20260929 02:55:45.851418 24243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:55:45 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650545, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:55:45.854310 24243 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:55:45.855693 24243 webserver.cc:460] Webserver started at http://127.8.57.3:42725/ using document root <none> and password file <none>
I20260929 02:55:45.856375 24243 fs_manager.cc:359] Metadata directory not provided
I20260929 02:55:45.856489 24243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:55:45.856774 24243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:55:45.859617 24243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "62ef8d45ddd14036b38db84a76ceb413"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "94caff26155b72559dbc400ae999dbc6"
server_key_iv: "0747acebfafaa98fee66966b463d2781"
server_key_version: "encryptionkey@0"
I20260929 02:55:45.860394 24243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "62ef8d45ddd14036b38db84a76ceb413"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "94caff26155b72559dbc400ae999dbc6"
server_key_iv: "0747acebfafaa98fee66966b463d2781"
server_key_version: "encryptionkey@0"
I20260929 02:55:45.865034 24243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:55:45.868268 24258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:45.869448 24243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 02:55:45.869586 24243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "62ef8d45ddd14036b38db84a76ceb413"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "94caff26155b72559dbc400ae999dbc6"
server_key_iv: "0747acebfafaa98fee66966b463d2781"
server_key_version: "encryptionkey@0"
I20260929 02:55:45.869747 24243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:55:45.888679 24243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:55:45.892560 24243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:55:45.892812 24243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:55:45.893707 24243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:55:45.894999 24243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:55:45.895072 24243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:45.895196 24243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:55:45.895246 24243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:55:45.924132 24243 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:40683
I20260929 02:55:45.924161 24370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:40683 every 8 connection(s)
I20260929 02:55:45.925263 24243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:55:45.926889 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24243
I20260929 02:55:45.927050 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:55:45.927594 8420 external_mini_cluster.cc:1428] Setting key bee0d50c3f71587fb7966a20c3b3f1ec
Sep 29 02:55:45 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650545, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:55:45.943791 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.927679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:60547 (local address 127.8.57.62:41191)
0929 02:55:45.927847 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:45.927851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:45.929156 (+ 1305us) server_negotiation.cc:407] Connection header received
0929 02:55:45.931203 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:45.931207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:45.931268 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:45.931336 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:45.932739 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.933465 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.935010 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.935416 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.935527 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:45.938670 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:45.938675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:45.938678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:45.938709 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:45.940956 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:45.941693 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:45.941697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:45.941699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:45.941870 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:45.942344 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:45.942348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:45.942351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:45.942669 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:45.942779 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:45.943377 (+ 598us) server_negotiation.cc:299] Negotiation successful
0929 02:55:45.943551 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:55:45.944552 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.928258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:60547)
0929 02:55:45.928991 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:45.929044 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:45.930974 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:45.931514 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:45.931521 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:45.932064 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:55:45.932584 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.932601 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.933633 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.933639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.934883 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.934895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.935620 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.935627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.935845 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:45.936434 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:55:45.936459 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:55:45.938466 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:55:45.941085 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:45.941099 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:45.941113 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:45.941576 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:45.941991 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:55:45.941995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:55:45.941999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:55:45.942226 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:55:45.942775 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:55:45.942781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:55:45.943051 (+ 270us) client_negotiation.cc:769] Sending connection context
0929 02:55:45.943256 (+ 205us) client_negotiation.cc:240] Negotiation successful
0929 02:55:45.943577 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":282,"threads_started":1}
I20260929 02:55:45.945680 24371 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41191
I20260929 02:55:45.945935 24371 heartbeater.cc:464] Registering TS with master...
I20260929 02:55:45.946578 24371 heartbeater.cc:511] Master 127.8.57.62:41191 requested a full tablet report, sending...
I20260929 02:55:45.947817 23915 ts_manager.cc:194] Registered new tserver with Master: 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3:40683)
I20260929 02:55:45.948877 23915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:60547
I20260929 02:55:45.954638 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:55:45.969913 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.958972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.8.57.62:41191)
0929 02:55:45.959104 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:45.959108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:45.959515 (+ 407us) server_negotiation.cc:407] Connection header received
0929 02:55:45.959662 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:45.959665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:45.959720 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:45.959797 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:45.961144 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.961821 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.963056 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.963423 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.963506 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:45.965545 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:45.965549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:45.965552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:45.965579 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:45.967904 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:45.968481 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:45.968484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:45.968486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:45.968589 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:45.968853 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:45.968856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:45.968858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:45.969133 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:45.969306 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:45.969544 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 02:55:45.969707 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:55:45.973425 23915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:55:45.980007 23915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:55:45.980633 24104 heartbeater.cc:503] Master 127.8.57.62:41191 was elected leader, sending a full tablet report...
I20260929 02:55:46.002172 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.991000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43679 (local address 127.0.0.1:48350)
0929 02:55:45.992009 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:45.992048 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:45.992250 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:45.994486 (+ 2236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:45.994491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:45.994516 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:45.995083 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.995100 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.997038 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.997043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.999620 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.999639 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.001386 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.001393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.001540 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.001571 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:55:46.001658 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 02:55:46.001767 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":190,"thread_start_us":1972,"threads_started":1}
I20260929 02:55:46.002223 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.992202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.8.57.3:40683)
0929 02:55:45.993799 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:45.993806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:45.993893 (+ 87us) server_negotiation.cc:407] Connection header received
0929 02:55:45.993983 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:45.993991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:45.994208 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:45.994391 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:45.995468 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.996818 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.999838 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.000844 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.000960 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.001198 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.001756 (+ 558us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.001918 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1445,"thread_start_us":1274,"threads_started":1}
I20260929 02:55:46.002235 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.991241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.8.57.2:43679)
0929 02:55:45.993421 (+ 2180us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:45.993431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:45.993460 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:55:45.993558 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:45.993566 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:45.993808 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:45.995294 (+ 1486us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:45.995527 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.996865 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.999822 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.001204 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.001313 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.001492 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.001733 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.001912 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2030,"thread_start_us":1941,"threads_started":1}
I20260929 02:55:46.003063 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.991322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:40683 (local address 127.0.0.1:46132)
0929 02:55:45.992863 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:45.992881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:45.992985 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:45.994440 (+ 1455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:45.994449 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:45.994484 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:45.995116 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.995129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.996940 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.996945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.999682 (+ 2737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.999692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.001015 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.001021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.001219 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.001271 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 02:55:46.001400 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 02:55:46.001612 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"mutex_wait_us":196,"thread_start_us":1219,"threads_started":1}
I20260929 02:55:46.003438 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.991620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:52530)
0929 02:55:45.995132 (+ 3512us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:45.995153 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:45.995258 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:45.995929 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:45.995934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:45.995958 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:45.996283 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.996294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.997707 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.997712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:45.999634 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:45.999646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.002611 (+ 2965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.002618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.002779 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.002812 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:55:46.002915 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 02:55:46.003041 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3379,"mutex_wait_us":24,"thread_start_us":3157,"threads_started":1}
I20260929 02:55:46.003440 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:45.992731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.8.57.1:46703)
0929 02:55:45.994380 (+ 1649us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:45.994387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:45.995383 (+ 996us) server_negotiation.cc:407] Connection header received
0929 02:55:45.995487 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:45.995494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:45.995712 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:45.995927 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:45.996431 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:45.997572 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:45.999774 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.001189 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.001314 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.001517 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.002977 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.003130 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1499,"thread_start_us":1336,"threads_started":1}
I20260929 02:55:46.006055 24173 tablet_service.cc:1467] Processing CreateTablet for tablet 2820e968a4704c7faf692820ffdd1336 (DEFAULT_TABLE table=test-table [id=05d50aadcd6e4308b27a282665f86d88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:46.007242 24173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2820e968a4704c7faf692820ffdd1336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:46.006804 24039 tablet_service.cc:1467] Processing CreateTablet for tablet 2820e968a4704c7faf692820ffdd1336 (DEFAULT_TABLE table=test-table [id=05d50aadcd6e4308b27a282665f86d88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:46.006810 24306 tablet_service.cc:1467] Processing CreateTablet for tablet 2820e968a4704c7faf692820ffdd1336 (DEFAULT_TABLE table=test-table [id=05d50aadcd6e4308b27a282665f86d88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:55:46.008183 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2820e968a4704c7faf692820ffdd1336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:46.008191 24039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2820e968a4704c7faf692820ffdd1336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:55:46.018025 24389 tablet_bootstrap.cc:492] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Bootstrap starting.
I20260929 02:55:46.021694 24390 tablet_bootstrap.cc:492] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Bootstrap starting.
I20260929 02:55:46.021955 24389 tablet_bootstrap.cc:654] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:46.023138 24391 tablet_bootstrap.cc:492] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Bootstrap starting.
I20260929 02:55:46.023204 24389 log.cc:824] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:46.024904 24390 tablet_bootstrap.cc:654] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:46.026101 24391 tablet_bootstrap.cc:654] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Neither blocks nor log segments found. Creating new log.
I20260929 02:55:46.026304 24390 log.cc:824] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:46.026492 24389 tablet_bootstrap.cc:492] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: No bootstrap required, opened a new log
I20260929 02:55:46.026787 24389 ts_tablet_manager.cc:1397] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:55:46.027338 24391 log.cc:824] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Log is configured to *not* fsync() on all Append() calls
I20260929 02:55:46.029183 24390 tablet_bootstrap.cc:492] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: No bootstrap required, opened a new log
I20260929 02:55:46.029422 24390 ts_tablet_manager.cc:1397] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:55:46.029594 24391 tablet_bootstrap.cc:492] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: No bootstrap required, opened a new log
I20260929 02:55:46.029806 24391 ts_tablet_manager.cc:1397] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260929 02:55:46.036736 24389 raft_consensus.cc:348] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.036943 24391 raft_consensus.cc:348] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.037149 24389 raft_consensus.cc:374] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:46.037240 24389 raft_consensus.cc:729] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c43e658a90f4bca833bda8db8a8dcbf, State: Initialized, Role: FOLLOWER
I20260929 02:55:46.037309 24391 raft_consensus.cc:374] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:46.037426 24391 raft_consensus.cc:729] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62ef8d45ddd14036b38db84a76ceb413, State: Initialized, Role: FOLLOWER
I20260929 02:55:46.037843 24389 consensus_queue.cc:260] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.037940 24391 consensus_queue.cc:260] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.038902 24371 heartbeater.cc:503] Master 127.8.57.62:41191 was elected leader, sending a full tablet report...
I20260929 02:55:46.039342 24238 heartbeater.cc:503] Master 127.8.57.62:41191 was elected leader, sending a full tablet report...
I20260929 02:55:46.040021 24389 ts_tablet_manager.cc:1428] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 02:55:46.040617 24391 ts_tablet_manager.cc:1428] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Time spent starting tablet: real 0.011s user 0.004s sys 0.007s
I20260929 02:55:46.040412 24390 raft_consensus.cc:348] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.040856 24390 raft_consensus.cc:374] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:55:46.040966 24390 raft_consensus.cc:729] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8188632b7354b4f8c75ee098bb17b6e, State: Initialized, Role: FOLLOWER
I20260929 02:55:46.041518 24390 consensus_queue.cc:260] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.043051 24390 ts_tablet_manager.cc:1428] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 02:55:46.159576 24397 raft_consensus.cc:482] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:55:46.159862 24397 raft_consensus.cc:504] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.161505 24397 leader_election.cc:288] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679), 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3:40683)
I20260929 02:55:46.170060 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.161874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:36769 (local address 127.8.57.3:40683)
0929 02:55:46.162071 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:46.162077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:46.162705 (+ 628us) server_negotiation.cc:407] Connection header received
0929 02:55:46.162896 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:46.162901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:46.162977 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:46.163068 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:46.163728 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.164876 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.167685 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.168901 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.169022 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.169119 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.169671 (+ 552us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.169807 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:55:46.170058 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.161910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:56397 (local address 127.8.57.2:43679)
0929 02:55:46.162075 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:46.162080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:46.162749 (+ 669us) server_negotiation.cc:407] Connection header received
0929 02:55:46.162910 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:46.162915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:46.162982 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:46.163071 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:55:46.163800 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.164877 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.167684 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.168901 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.169021 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.169129 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.169671 (+ 542us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.169812 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:55:46.170389 24399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.161773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:40683 (local address 127.8.57.1:36769)
0929 02:55:46.162583 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:46.162606 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:46.162759 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:46.163100 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:46.163105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:46.163152 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:46.163564 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:46.163579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.165007 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.165012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.167518 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:46.167534 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.169119 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.169130 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.169440 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.169509 (+ 69us) client_negotiation.cc:769] Sending connection context
0929 02:55:46.169600 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 02:55:46.169792 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":241,"spinlock_wait_cycles":23936,"thread_start_us":511,"threads_started":1}
I20260929 02:55:46.170360 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.161876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43679 (local address 127.8.57.1:56397)
0929 02:55:46.162648 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0929 02:55:46.162661 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:55:46.162804 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:55:46.163100 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:55:46.163105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:55:46.163130 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:55:46.163617 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:46.163627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.165005 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.165011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.167526 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:55:46.167535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.169147 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.169151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:55:46.169433 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.169508 (+ 75us) client_negotiation.cc:769] Sending connection context
0929 02:55:46.169601 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 02:55:46.169792 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":284,"spinlock_wait_cycles":19712,"thread_start_us":379,"threads_started":1}
I20260929 02:55:46.171334 24193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2820e968a4704c7faf692820ffdd1336" candidate_uuid: "c8188632b7354b4f8c75ee098bb17b6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" is_pre_election: true
I20260929 02:55:46.171361 24326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2820e968a4704c7faf692820ffdd1336" candidate_uuid: "c8188632b7354b4f8c75ee098bb17b6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62ef8d45ddd14036b38db84a76ceb413" is_pre_election: true
I20260929 02:55:46.171780 24193 raft_consensus.cc:2454] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8188632b7354b4f8c75ee098bb17b6e in term 0.
I20260929 02:55:46.171782 24326 raft_consensus.cc:2454] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8188632b7354b4f8c75ee098bb17b6e in term 0.
I20260929 02:55:46.172420 23993 leader_election.cc:302] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c43e658a90f4bca833bda8db8a8dcbf, c8188632b7354b4f8c75ee098bb17b6e; no voters:
I20260929 02:55:46.172791 24397 raft_consensus.cc:2781] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:55:46.172927 24397 raft_consensus.cc:482] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:55:46.173030 24397 raft_consensus.cc:3037] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:46.175107 24397 raft_consensus.cc:504] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.175650 24397 leader_election.cc:288] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [CANDIDATE]: Term 1 election: Requested vote from peers 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679), 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3:40683)
I20260929 02:55:46.176062 24193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2820e968a4704c7faf692820ffdd1336" candidate_uuid: "c8188632b7354b4f8c75ee098bb17b6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
I20260929 02:55:46.176096 24326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2820e968a4704c7faf692820ffdd1336" candidate_uuid: "c8188632b7354b4f8c75ee098bb17b6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62ef8d45ddd14036b38db84a76ceb413"
I20260929 02:55:46.176262 24193 raft_consensus.cc:3037] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:55:46.176308 24326 raft_consensus.cc:3037] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:55:46.176658 24372 tablet.cc:2369] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:46.179344 24193 raft_consensus.cc:2454] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8188632b7354b4f8c75ee098bb17b6e in term 1.
I20260929 02:55:46.179344 24326 raft_consensus.cc:2454] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8188632b7354b4f8c75ee098bb17b6e in term 1.
I20260929 02:55:46.179718 23995 leader_election.cc:302] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62ef8d45ddd14036b38db84a76ceb413, c8188632b7354b4f8c75ee098bb17b6e; no voters:
I20260929 02:55:46.180085 24397 raft_consensus.cc:2781] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:55:46.180495 24397 raft_consensus.cc:686] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 1 LEADER]: Becoming Leader. State: Replica: c8188632b7354b4f8c75ee098bb17b6e, State: Running, Role: LEADER
I20260929 02:55:46.180961 24397 consensus_queue.cc:237] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } }
I20260929 02:55:46.185045 23913 catalog_manager.cc:5495] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e reported cstate change: term changed from 0 to 1, leader changed from <none> to c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1). New cstate: current_term: 1 leader_uuid: "c8188632b7354b4f8c75ee098bb17b6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } health_report { overall_health: HEALTHY } } }
W20260929 02:55:46.202072 24105 tablet.cc:2369] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:55:46.204990 24239 tablet.cc:2369] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:55:46.242102 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.236992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.8.57.1:46703)
0929 02:55:46.237210 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:46.237215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:46.237232 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:55:46.237343 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:46.237347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:46.237437 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:46.237552 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:55:46.237841 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.238577 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.240106 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.240508 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.240600 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.241070 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:55:46.241555 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:55:46.241666 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.241786 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.241851 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:55:46.249711 24326 raft_consensus.cc:1261] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Refusing update from remote peer c8188632b7354b4f8c75ee098bb17b6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:46.249711 24193 raft_consensus.cc:1261] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Refusing update from remote peer c8188632b7354b4f8c75ee098bb17b6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:55:46.250674 24397 consensus_queue.cc:1035] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:46.250952 24400 consensus_queue.cc:1035] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:55:46.262753 24403 mvcc.cc:205] Tried to move back new op lower bound from 7334504637429399552 to 7334504637160566784. Current Snapshot: MvccSnapshot[applied={T|T < 7334504637429399552}]
I20260929 02:55:46.264382 24405 mvcc.cc:205] Tried to move back new op lower bound from 7334504637429399552 to 7334504637160566784. Current Snapshot: MvccSnapshot[applied={T|T < 7334504637429399552}]
I20260929 02:55:46.277572 23913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:46.277824 23913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:55:46.280962 23913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Sending DeleteTablet for 3 replicas of tablet 2820e968a4704c7faf692820ffdd1336
I20260929 02:55:46.281968 24039 tablet_service.cc:1514] Processing DeleteTablet for tablet 2820e968a4704c7faf692820ffdd1336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:46 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:52530
I20260929 02:55:46.281962 24306 tablet_service.cc:1514] Processing DeleteTablet for tablet 2820e968a4704c7faf692820ffdd1336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:46 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:46132
I20260929 02:55:46.281981 24173 tablet_service.cc:1514] Processing DeleteTablet for tablet 2820e968a4704c7faf692820ffdd1336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:55:46 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:48350
I20260929 02:55:46.284155 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23976
I20260929 02:55:46.293358 24414 tablet_replica.cc:330] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: stopping tablet replica
I20260929 02:55:46.293993 24414 raft_consensus.cc:2229] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:55:46.294842 24414 raft_consensus.cc:2258] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:46.295650 24413 tablet_replica.cc:330] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: stopping tablet replica
I20260929 02:55:46.296121 24413 raft_consensus.cc:2229] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:46.296669 24413 raft_consensus.cc:2258] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:46.297299 24414 ts_tablet_manager.cc:1905] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:46.297322 24415 tablet_replica.cc:330] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: stopping tablet replica
I20260929 02:55:46.297693 24415 raft_consensus.cc:2229] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:55:46.298240 24415 raft_consensus.cc:2258] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:55:46.298748 24413 ts_tablet_manager.cc:1905] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:46.300271 24415 ts_tablet_manager.cc:1905] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:55:46.305547 24413 ts_tablet_manager.cc:1918] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:46.305776 24413 log.cc:1196] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/2820e968a4704c7faf692820ffdd1336
I20260929 02:55:46.306354 24413 ts_tablet_manager.cc:1939] T 2820e968a4704c7faf692820ffdd1336 P 62ef8d45ddd14036b38db84a76ceb413: Deleting consensus metadata
I20260929 02:55:46.307078 24415 ts_tablet_manager.cc:1918] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:46.307291 24415 log.cc:1196] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/2820e968a4704c7faf692820ffdd1336
I20260929 02:55:46.307780 24415 ts_tablet_manager.cc:1939] T 2820e968a4704c7faf692820ffdd1336 P 9c43e658a90f4bca833bda8db8a8dcbf: Deleting consensus metadata
I20260929 02:55:46.308354 23902 catalog_manager.cc:4841] TS 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3:40683): tablet 2820e968a4704c7faf692820ffdd1336 (table test-table [id=05d50aadcd6e4308b27a282665f86d88]) successfully deleted
I20260929 02:55:46.309442 23900 catalog_manager.cc:4841] TS 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679): tablet 2820e968a4704c7faf692820ffdd1336 (table test-table [id=05d50aadcd6e4308b27a282665f86d88]) successfully deleted
I20260929 02:55:46.311331 24414 ts_tablet_manager.cc:1918] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:55:46.311479 24414 log.cc:1196] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/2820e968a4704c7faf692820ffdd1336
I20260929 02:55:46.311925 24414 ts_tablet_manager.cc:1939] T 2820e968a4704c7faf692820ffdd1336 P c8188632b7354b4f8c75ee098bb17b6e: Deleting consensus metadata
I20260929 02:55:46.313179 23901 catalog_manager.cc:4841] TS c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1:46703): tablet 2820e968a4704c7faf692820ffdd1336 (table test-table [id=05d50aadcd6e4308b27a282665f86d88]) successfully deleted
Sep 29 02:55:46 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:55:46.327188 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.285311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.8.57.1:46703)
0929 02:55:46.285523 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:46.285529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:46.307720 (+ 22191us) server_negotiation.cc:407] Connection header received
0929 02:55:46.308238 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:46.308244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:46.308334 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:46.308407 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:46.310145 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.311406 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.313388 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.313829 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.313918 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.318774 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:46.318788 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:46.318805 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:46.318858 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:46.324266 (+ 5408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:46.325080 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:46.325087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:46.325106 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:46.325258 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:46.325654 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:46.325659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:46.325663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:46.326027 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:46.326456 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.326605 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.326830 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:55:46.327800 24099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:46.440611 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23976
I20260929 02:55:46.455310 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 24110
Sep 29 02:55:46 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:55:46.470888 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.456007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.8.57.2:43679)
0929 02:55:46.456573 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:46.456579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:46.456595 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:55:46.456649 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:46.456653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:46.456711 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:46.456801 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:46.458256 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.459044 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.460481 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.460889 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.460994 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.465287 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:46.465298 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:46.465310 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:46.465345 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:46.468217 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:46.468918 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:46.468925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:46.468928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:46.469054 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:46.469402 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:46.469406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:46.469408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:46.469788 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:46.470046 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.470363 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.470562 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438}
I20260929 02:55:46.471446 24233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:46.559612 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 24110
I20260929 02:55:46.572423 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 24243
Sep 29 02:55:46 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:55:46.586974 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:55:46.573137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.8.57.3:40683)
0929 02:55:46.573331 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:55:46.573338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:55:46.573413 (+ 75us) server_negotiation.cc:407] Connection header received
0929 02:55:46.573595 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:55:46.573600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:55:46.573679 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:55:46.573784 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:55:46.575127 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.575865 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.577165 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:55:46.577544 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:55:46.577655 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:55:46.581801 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:55:46.581816 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:55:46.581831 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:55:46.581869 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:55:46.584437 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:46.585063 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:46.585069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:46.585072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:46.585207 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:55:46.585530 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:55:46.585534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:55:46.585536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:55:46.585871 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:55:46.586121 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 02:55:46.586446 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 02:55:46.586657 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:55:46.587590 24366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:46.674299 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 24243
I20260929 02:55:46.686911 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 23883
I20260929 02:55:46.688014 23945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:55:46.794902 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 23883
I20260929 02:56:06.812932 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41191
--webserver_interface=127.8.57.62
--webserver_port=34399
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:07.128965 24421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:07.129226 24421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:07.129364 24421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:07.137583 24421 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 02:56:07.137760 24421 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 02:56:07.137850 24421 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 02:56:07.137944 24421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:56:07.138036 24421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:07.138145 24421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:07.138223 24421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:56:07.138311 24421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:56:07.150559 24421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=34399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:07.151778 24421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:07.153173 24421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:07.174110 24427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:07.174167 24428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:07.174146 24430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:07.174256 24421 server_base.cc:1034] running on GCE node
I20260929 02:56:07.175439 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:07.176959 24421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:07.178169 24421 hybrid_clock.cc:648] HybridClock initialized: now 1790650567178104 us; error 37 us; skew 500 ppm
I20260929 02:56:07.178539 24421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:07 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:07.182036 24421 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:56:07.183670 24421 webserver.cc:460] Webserver started at http://127.8.57.62:34399/ using document root <none> and password file <none>
I20260929 02:56:07.184455 24421 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:07.184567 24421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:07.190228 24421 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:56:07.193015 24437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:07.194252 24421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:07.194398 24421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "4e5ea13a287842959dfd3d14eb0b5fab"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "89ae472867c2b3904416bbce2c983e1f"
server_key_iv: "84af357e4d476f5e1b580c51044eaaa2"
server_key_version: "encryptionkey@0"
I20260929 02:56:07.195359 24421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:07.230281 24421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:07.234588 24421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:07.234867 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:07.252621 24421 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41191
I20260929 02:56:07.252671 24488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41191 every 8 connection(s)
I20260929 02:56:07.254205 24421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:56:07.261248 24489 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:56:07.264310 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24421
I20260929 02:56:07.264503 24489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap starting.
I20260929 02:56:07.265141 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:46703
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:41191
--webserver_port=44789
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:56:07.283537 24489 log.cc:824] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:07.295696 24489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:56:07.296278 24489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap complete.
I20260929 02:56:07.305543 24489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:07.306411 24489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5ea13a287842959dfd3d14eb0b5fab, State: Initialized, Role: FOLLOWER
I20260929 02:56:07.306981 24489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:07.307207 24489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:56:07.307333 24489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:56:07.307479 24489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:56:07.311251 24489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:07.311729 24489 leader_election.cc:302] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e5ea13a287842959dfd3d14eb0b5fab; no voters:
I20260929 02:56:07.312357 24489 leader_election.cc:288] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:56:07.312536 24493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:56:07.313638 24493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 LEADER]: Becoming Leader. State: Replica: 4e5ea13a287842959dfd3d14eb0b5fab, State: Running, Role: LEADER
I20260929 02:56:07.314245 24493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:07.314565 24489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:56:07.316711 24494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } } }
I20260929 02:56:07.316681 24495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e5ea13a287842959dfd3d14eb0b5fab. Latest consensus state: current_term: 2 leader_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } } }
I20260929 02:56:07.317152 24494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:07.317157 24495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:07.318015 24499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:56:07.338088 24499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:56:07.339562 24499 catalog_manager.cc:1259] Loaded cluster ID: d841d6af8e6d477d8c922e9a8c15087c
I20260929 02:56:07.339720 24499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:56:07.343729 24499 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:56:07.345314 24499 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Loaded TSK: 0
I20260929 02:56:07.380048 24499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Generated new TSK 1
I20260929 02:56:07.380604 24499 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:56:07.618782 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:07.619061 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:07.619174 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:07.626497 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:07.626601 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:07.626739 24491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:56:07.636356 24491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:46703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=44789
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:07.637563 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:07.638793 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:07.652621 24517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:07.652848 24516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:07.652598 24519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:07.653255 24491 server_base.cc:1034] running on GCE node
I20260929 02:56:07.654100 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:07.655005 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:07.656208 24491 hybrid_clock.cc:648] HybridClock initialized: now 1790650567656143 us; error 58 us; skew 500 ppm
I20260929 02:56:07.656498 24491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:07 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:07.659667 24491 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:56:07.661134 24491 webserver.cc:460] Webserver started at http://127.8.57.1:44789/ using document root <none> and password file <none>
I20260929 02:56:07.661792 24491 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:07.661926 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:07.666987 24491 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:56:07.669544 24526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:07.670810 24491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:56:07.670953 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "c8188632b7354b4f8c75ee098bb17b6e"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "48764635a77c90f580dae7686df3305c"
server_key_iv: "0b3020141bf6ab7ede22cae5bd42713b"
server_key_version: "encryptionkey@0"
I20260929 02:56:07.671943 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:07.686681 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:07.690583 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:07.690824 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:07.691756 24491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:07.693233 24491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:07.693337 24491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:07.693436 24491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:07.693511 24491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:07.726542 24491 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:46703
I20260929 02:56:07.726598 24638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:46703 every 8 connection(s)
I20260929 02:56:07.727947 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:56:07.736378 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24491
I20260929 02:56:07.736949 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:43679
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:41191
--webserver_port=33705
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:56:07 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:07.751186 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:07.730655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:50619)
0929 02:56:07.731412 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:07.731462 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:07.733137 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:07.734412 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:07.734419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:07.734912 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:07.735416 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:07.735432 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:07.737042 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:07.737047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:07.738881 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:07.738896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:07.739928 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:07.739935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:07.740190 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:07.740889 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:07.740914 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:07.743526 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:07.746818 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:07.746828 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:07.746845 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:07.747283 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:07.747743 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:07.747749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:07.747753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:07.747998 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:07.748718 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:07.748725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:07.749084 (+ 359us) client_negotiation.cc:769] Sending connection context
0929 02:56:07.749312 (+ 228us) client_negotiation.cc:240] Negotiation successful
0929 02:56:07.749865 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":309,"threads_started":1}
I20260929 02:56:07.751186 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:07.730577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:50619 (local address 127.8.57.62:41191)
0929 02:56:07.731159 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:07.731170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:07.731593 (+ 423us) server_negotiation.cc:407] Connection header received
0929 02:56:07.733646 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:07.733658 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:07.733985 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:07.734273 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:07.735583 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:07.736867 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:07.739040 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:07.739686 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:07.739827 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:07.743820 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:07.743830 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:07.743845 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:07.743915 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:07.746655 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:07.747442 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:07.747449 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:07.747455 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:07.747630 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:07.748103 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:07.748107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:07.748110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:07.748586 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:07.748774 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:07.749568 (+ 794us) server_negotiation.cc:299] Negotiation successful
0929 02:56:07.749951 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":256,"threads_started":1}
I20260929 02:56:07.754012 24639 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41191
I20260929 02:56:07.754360 24639 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:07.755262 24639 heartbeater.cc:511] Master 127.8.57.62:41191 requested a full tablet report, sending...
I20260929 02:56:07.757543 24454 ts_manager.cc:194] Registered new tserver with Master: c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1:46703)
I20260929 02:56:07.758991 24454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:50619
W20260929 02:56:08.051710 24643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:08.052026 24643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:08.052171 24643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:08.059509 24643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:08.059615 24643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:08.059762 24643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:56:08.069429 24643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:43679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=33705
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:08.070680 24643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:08.071928 24643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:08.084367 24651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:08.084434 24649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:08.084405 24648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:08.085386 24643 server_base.cc:1034] running on GCE node
I20260929 02:56:08.085942 24643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:08.086819 24643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:08.087988 24643 hybrid_clock.cc:648] HybridClock initialized: now 1790650568087937 us; error 42 us; skew 500 ppm
I20260929 02:56:08.088284 24643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:08 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:08.091092 24643 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:56:08.092377 24643 webserver.cc:460] Webserver started at http://127.8.57.2:33705/ using document root <none> and password file <none>
I20260929 02:56:08.092990 24643 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:08.093101 24643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:08.098296 24643 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:56:08.100975 24658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:08.102272 24643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:08.102406 24643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "c4065c86f928a53e917bdb87dd56e436"
server_key_iv: "500a7f1036640e8ace4f6716e47e04e6"
server_key_version: "encryptionkey@0"
I20260929 02:56:08.103295 24643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:08.118548 24643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:08.121939 24643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:08.122146 24643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:08.122968 24643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:08.124182 24643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:08.124262 24643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:08.124346 24643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:08.124401 24643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:08.152359 24643 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:43679
I20260929 02:56:08.152411 24770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:43679 every 8 connection(s)
I20260929 02:56:08.153664 24643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:56:08.159469 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24643
I20260929 02:56:08.160235 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:40683
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:41191
--webserver_port=42725
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:56:08 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:08.174091 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.156045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:50317 (local address 127.8.57.62:41191)
0929 02:56:08.156239 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.156245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.157392 (+ 1147us) server_negotiation.cc:407] Connection header received
0929 02:56:08.159095 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.159099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.159166 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.159248 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:08.161122 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.161813 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.163754 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.164185 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.164278 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.168029 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:08.168036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:08.168040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:08.168076 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:08.171240 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.171923 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.171929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.171932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.172089 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.172490 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.172495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.172499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.172771 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:08.172901 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.173533 (+ 632us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.173754 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:56:08.175196 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.156546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:50317)
0929 02:56:08.157247 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:08.157282 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:08.158851 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:08.159517 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:08.159529 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:08.160229 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:08.160959 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.160985 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.161956 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.161961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.163619 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.163633 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.164385 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.164391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.164654 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.165284 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:08.165309 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:08.167774 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:08.171372 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:08.171380 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:08.171396 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:08.171784 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:08.172194 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:08.172199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:08.172203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:08.172380 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:08.172889 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:08.172897 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:08.173199 (+ 302us) client_negotiation.cc:769] Sending connection context
0929 02:56:08.173415 (+ 216us) client_negotiation.cc:240] Negotiation successful
0929 02:56:08.173907 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":306,"threads_started":1}
I20260929 02:56:08.176772 24771 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41191
I20260929 02:56:08.177035 24771 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:08.177659 24771 heartbeater.cc:511] Master 127.8.57.62:41191 requested a full tablet report, sending...
I20260929 02:56:08.178963 24454 ts_manager.cc:194] Registered new tserver with Master: 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679)
I20260929 02:56:08.179795 24454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:50317
W20260929 02:56:08.474903 24774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:08.475167 24774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:08.475299 24774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:08.482263 24774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:08.482368 24774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:08.482520 24774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:56:08.491671 24774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:40683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=42725
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:08.492798 24774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:08.493969 24774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:08.505789 24779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:08.505851 24782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:08.505828 24780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:08.506556 24774 server_base.cc:1034] running on GCE node
I20260929 02:56:08.507148 24774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:08.508044 24774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:08.509220 24774 hybrid_clock.cc:648] HybridClock initialized: now 1790650568509180 us; error 34 us; skew 500 ppm
I20260929 02:56:08.509496 24774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:08 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:08.512377 24774 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:56:08.513633 24774 webserver.cc:460] Webserver started at http://127.8.57.3:42725/ using document root <none> and password file <none>
I20260929 02:56:08.514271 24774 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:08.514389 24774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:08.519364 24774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:56:08.521821 24789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:08.522994 24774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:56:08.523145 24774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "62ef8d45ddd14036b38db84a76ceb413"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "94caff26155b72559dbc400ae999dbc6"
server_key_iv: "0747acebfafaa98fee66966b463d2781"
server_key_version: "encryptionkey@0"
I20260929 02:56:08.524089 24774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:08.568878 24774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:08.572908 24774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:08.573156 24774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:08.574061 24774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:08.575312 24774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:08.575403 24774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:08.575501 24774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:08.575577 24774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:08.604163 24774 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:40683
I20260929 02:56:08.604226 24901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:40683 every 8 connection(s)
I20260929 02:56:08.605451 24774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:56:08.615046 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24774
Sep 29 02:56:08 dist-test-slave-1dkt krb5kdc[23870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:08.625272 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.607906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:54717 (local address 127.8.57.62:41191)
0929 02:56:08.608072 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.608076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.609171 (+ 1095us) server_negotiation.cc:407] Connection header received
0929 02:56:08.610930 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.610935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.611014 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.611102 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:08.612558 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.613253 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.614816 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.615273 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.615374 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.619067 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:08.619075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:08.619079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:08.619108 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:08.622087 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.622888 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.622895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.622899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.623063 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.623566 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.623571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.623575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.623844 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:08.624014 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.624652 (+ 638us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.624861 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 02:56:08.626214 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.608314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:54717)
0929 02:56:08.609006 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:08.609049 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:08.610646 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:08.611303 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:08.611311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:08.611804 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:08.612407 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.612424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.613377 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.613380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.614694 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.614709 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.615505 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.615513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.615749 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.616435 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:08.616465 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:08.618860 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:08.622218 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:08.622226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:08.622241 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:08.622724 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:08.623192 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:08.623197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:08.623201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:08.623442 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:08.623978 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:08.623986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:08.624307 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 02:56:08.624540 (+ 233us) client_negotiation.cc:240] Negotiation successful
0929 02:56:08.625025 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":260,"threads_started":1}
I20260929 02:56:08.627547 24902 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41191
I20260929 02:56:08.627844 24902 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:08.628568 24902 heartbeater.cc:511] Master 127.8.57.62:41191 requested a full tablet report, sending...
I20260929 02:56:08.629904 24454 ts_manager.cc:194] Registered new tserver with Master: 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3:40683)
I20260929 02:56:08.630023 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.616082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.8.57.62:41191)
0929 02:56:08.616956 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.616960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.616973 (+ 13us) server_negotiation.cc:407] Connection header received
0929 02:56:08.617010 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.617014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.617073 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.617151 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:08.618855 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.619709 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.621568 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.621985 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.622093 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.624835 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:08.624840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:08.624843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:08.624870 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:08.627534 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.628244 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.628249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.628253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.628359 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.628681 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.628685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.628688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.628941 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:08.629171 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.629466 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.629678 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":653,"threads_started":1}
I20260929 02:56:08.630877 24454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:54717
I20260929 02:56:08.632195 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:56:08.637888 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.633059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.8.57.62:41191)
0929 02:56:08.633239 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.633243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.633649 (+ 406us) server_negotiation.cc:407] Connection header received
0929 02:56:08.633803 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.633806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.633919 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.634030 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:56:08.634368 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.635024 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.636483 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.636855 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.636938 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.637318 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:56:08.637480 (+ 162us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 02:56:08.637715 (+ 235us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51558: token expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 02:56:08.637986 24906 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51558: token expired
W20260929 02:56:08.638456 24377 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.57.62:41191)
I20260929 02:56:08.649506 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.638856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.8.57.62:41191)
0929 02:56:08.639003 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.639008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.639024 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:56:08.639146 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.639151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.639201 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.639270 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:08.640450 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.641114 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.642405 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.642772 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.642874 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.645320 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:08.645325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:08.645328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:08.645357 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:08.647441 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.648083 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.648087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.648089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.648202 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:08.648464 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:08.648466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:08.648468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:08.648746 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:08.648909 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.649140 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.649289 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:56:08.652531 24454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:56:08.659566 24454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:56:08.697283 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.671839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.8.57.3:40683)
0929 02:56:08.687701 (+ 15862us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.687710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.690439 (+ 2729us) server_negotiation.cc:407] Connection header received
0929 02:56:08.690569 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.690575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.690760 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.690949 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:08.691420 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.692920 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.694667 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.695817 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.695997 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.696231 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.696802 (+ 571us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.696949 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15667,"thread_start_us":16891,"threads_started":1}
I20260929 02:56:08.697374 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.670981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:40683 (local address 127.0.0.1:57392)
0929 02:56:08.690316 (+ 19335us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:08.690335 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:08.690443 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:08.690940 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:08.690945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:08.690969 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:08.691264 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.691274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.693066 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.693071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.694539 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.694553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.696374 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.696381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.696572 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.696630 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 02:56:08.696755 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 02:56:08.696931 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19227,"mutex_wait_us":29,"thread_start_us":17304,"threads_started":1}
I20260929 02:56:08.697499 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.670447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43679 (local address 127.0.0.1:59888)
0929 02:56:08.686738 (+ 16291us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:08.686776 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:08.686973 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:08.689059 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:08.689067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:08.689098 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:08.689551 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.689566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.691504 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.691509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.694304 (+ 2795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.694317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.696685 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.696693 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.696853 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.696885 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:56:08.696944 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 02:56:08.697078 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16046,"mutex_wait_us":127,"thread_start_us":15900,"threads_started":1}
I20260929 02:56:08.698413 24914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.670795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.8.57.2:43679)
0929 02:56:08.688141 (+ 17346us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.688149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.688187 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:56:08.688298 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.688306 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.688583 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.689804 (+ 1221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:08.690015 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.691379 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.694730 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.696318 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.697705 (+ 1387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.697881 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.697954 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.698125 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17193,"thread_start_us":17094,"threads_started":1}
I20260929 02:56:08.698508 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.670617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40184)
0929 02:56:08.688561 (+ 17944us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:08.688576 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:08.688766 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:08.689544 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:08.689547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:08.689565 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:08.689848 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.689854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.691694 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.691698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.694303 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:08.694317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.697315 (+ 2998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.697321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:08.697919 (+ 598us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.697949 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:56:08.698038 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 02:56:08.698148 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17818,"mutex_wait_us":105,"thread_start_us":19472,"threads_started":1}
I20260929 02:56:08.698624 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:08.671419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.8.57.1:46703)
0929 02:56:08.681642 (+ 10223us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:08.681652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:08.688669 (+ 7017us) server_negotiation.cc:407] Connection header received
0929 02:56:08.688934 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:08.688941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:08.689159 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:08.689291 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:08.689992 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.691307 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.694459 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:08.696855 (+ 2396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:08.696972 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:08.697144 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:08.698104 (+ 960us) server_negotiation.cc:299] Negotiation successful
0929 02:56:08.698256 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10030,"thread_start_us":9878,"threads_started":1}
I20260929 02:56:08.701386 24837 tablet_service.cc:1467] Processing CreateTablet for tablet f9e776250fbf40ec937807230076b652 (DEFAULT_TABLE table=test-table [id=14210f506f4a4f6782a023346a7deb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:08.701920 24706 tablet_service.cc:1467] Processing CreateTablet for tablet f9e776250fbf40ec937807230076b652 (DEFAULT_TABLE table=test-table [id=14210f506f4a4f6782a023346a7deb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:08.701988 24574 tablet_service.cc:1467] Processing CreateTablet for tablet f9e776250fbf40ec937807230076b652 (DEFAULT_TABLE table=test-table [id=14210f506f4a4f6782a023346a7deb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:08.702847 24837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e776250fbf40ec937807230076b652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:08.703069 24574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e776250fbf40ec937807230076b652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:08.703116 24706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e776250fbf40ec937807230076b652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:08.714604 24918 tablet_bootstrap.cc:492] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Bootstrap starting.
I20260929 02:56:08.717692 24918 tablet_bootstrap.cc:654] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:08.718468 24919 tablet_bootstrap.cc:492] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Bootstrap starting.
I20260929 02:56:08.720005 24920 tablet_bootstrap.cc:492] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Bootstrap starting.
I20260929 02:56:08.720206 24918 log.cc:824] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:08.721949 24919 tablet_bootstrap.cc:654] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:08.722837 24920 tablet_bootstrap.cc:654] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:08.723148 24919 log.cc:824] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:08.723348 24918 tablet_bootstrap.cc:492] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: No bootstrap required, opened a new log
I20260929 02:56:08.723609 24918 ts_tablet_manager.cc:1397] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260929 02:56:08.723839 24920 log.cc:824] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:08.726012 24919 tablet_bootstrap.cc:492] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: No bootstrap required, opened a new log
I20260929 02:56:08.726398 24919 ts_tablet_manager.cc:1397] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260929 02:56:08.726888 24920 tablet_bootstrap.cc:492] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: No bootstrap required, opened a new log
I20260929 02:56:08.727170 24920 ts_tablet_manager.cc:1397] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260929 02:56:08.735311 24918 raft_consensus.cc:348] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:08.735919 24918 raft_consensus.cc:374] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:08.736042 24918 raft_consensus.cc:729] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8188632b7354b4f8c75ee098bb17b6e, State: Initialized, Role: FOLLOWER
I20260929 02:56:08.736788 24918 consensus_queue.cc:260] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:08.737169 24920 raft_consensus.cc:348] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:08.737597 24920 raft_consensus.cc:374] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:08.737696 24920 raft_consensus.cc:729] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62ef8d45ddd14036b38db84a76ceb413, State: Initialized, Role: FOLLOWER
I20260929 02:56:08.737280 24919 raft_consensus.cc:348] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:08.737884 24919 raft_consensus.cc:374] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:08.738013 24919 raft_consensus.cc:729] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c43e658a90f4bca833bda8db8a8dcbf, State: Initialized, Role: FOLLOWER
I20260929 02:56:08.738286 24920 consensus_queue.cc:260] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:08.738646 24919 consensus_queue.cc:260] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:08.744894 24918 ts_tablet_manager.cc:1428] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Time spent starting tablet: real 0.021s user 0.012s sys 0.009s
I20260929 02:56:08.745236 24639 heartbeater.cc:503] Master 127.8.57.62:41191 was elected leader, sending a full tablet report...
I20260929 02:56:08.746989 24902 heartbeater.cc:503] Master 127.8.57.62:41191 was elected leader, sending a full tablet report...
I20260929 02:56:08.747717 24920 ts_tablet_manager.cc:1428] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Time spent starting tablet: real 0.020s user 0.012s sys 0.007s
I20260929 02:56:08.749236 24919 ts_tablet_manager.cc:1428] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Time spent starting tablet: real 0.023s user 0.018s sys 0.002s
I20260929 02:56:08.749773 24771 heartbeater.cc:503] Master 127.8.57.62:41191 was elected leader, sending a full tablet report...
W20260929 02:56:08.856932 24903 tablet.cc:2369] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:56:08.905504 24772 tablet.cc:2369] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:56:08.980340 24640 tablet.cc:2369] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:09.035653 24925 raft_consensus.cc:482] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:56:09.035971 24925 raft_consensus.cc:504] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:09.037629 24925 leader_election.cc:288] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679), c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1:46703)
I20260929 02:56:09.043817 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.038062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:43679 (local address 127.8.57.3:46561)
0929 02:56:09.038227 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.038247 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:09.038375 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:09.038734 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:09.038738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:09.038769 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:09.038960 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:09.038969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.040136 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.040140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:09.041991 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:09.042005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.043282 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.043286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:09.043422 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.043446 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:56:09.043514 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 02:56:09.043613 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":23}
I20260929 02:56:09.043941 24914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.038136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:46561 (local address 127.8.57.2:43679)
0929 02:56:09.038327 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:09.038333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:09.038354 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:56:09.038513 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:09.038519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:09.038603 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:09.038703 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:09.039075 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.040006 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.042128 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.043145 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.043270 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.043368 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:09.043556 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 02:56:09.043682 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:56:09.044796 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.038137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:49063 (local address 127.8.57.1:46703)
0929 02:56:09.038315 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:09.038321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:09.038938 (+ 617us) server_negotiation.cc:407] Connection header received
0929 02:56:09.039086 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:09.039092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:09.039167 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:09.039257 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:09.039687 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.040517 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.042581 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.043762 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.043910 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.044001 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:09.044527 (+ 526us) server_negotiation.cc:299] Negotiation successful
0929 02:56:09.044623 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:56:09.044930 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.038074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.8.57.3:49063)
0929 02:56:09.038835 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.038850 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:09.038965 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:09.039272 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:09.039274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:09.039292 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:09.039544 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:09.039551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.040639 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.040643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:09.042440 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:09.042450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.044245 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.044249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:09.044369 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.044397 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 02:56:09.044490 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 02:56:09.044608 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":238,"thread_start_us":292,"threads_started":1}
I20260929 02:56:09.044891 24726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e776250fbf40ec937807230076b652" candidate_uuid: "62ef8d45ddd14036b38db84a76ceb413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" is_pre_election: true
I20260929 02:56:09.045444 24726 raft_consensus.cc:2454] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62ef8d45ddd14036b38db84a76ceb413 in term 0.
I20260929 02:56:09.045840 24594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e776250fbf40ec937807230076b652" candidate_uuid: "62ef8d45ddd14036b38db84a76ceb413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8188632b7354b4f8c75ee098bb17b6e" is_pre_election: true
I20260929 02:56:09.046082 24791 leader_election.cc:302] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62ef8d45ddd14036b38db84a76ceb413, 9c43e658a90f4bca833bda8db8a8dcbf; no voters:
I20260929 02:56:09.046315 24594 raft_consensus.cc:2454] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62ef8d45ddd14036b38db84a76ceb413 in term 0.
I20260929 02:56:09.046449 24925 raft_consensus.cc:2781] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:56:09.046589 24925 raft_consensus.cc:482] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:56:09.046693 24925 raft_consensus.cc:3037] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:09.048933 24925 raft_consensus.cc:504] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:09.049463 24925 leader_election.cc:288] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [CANDIDATE]: Term 1 election: Requested vote from peers 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679), c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1:46703)
I20260929 02:56:09.049885 24594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e776250fbf40ec937807230076b652" candidate_uuid: "62ef8d45ddd14036b38db84a76ceb413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8188632b7354b4f8c75ee098bb17b6e"
I20260929 02:56:09.049872 24726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e776250fbf40ec937807230076b652" candidate_uuid: "62ef8d45ddd14036b38db84a76ceb413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
I20260929 02:56:09.050066 24594 raft_consensus.cc:3037] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:09.050076 24726 raft_consensus.cc:3037] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:09.053085 24726 raft_consensus.cc:2454] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62ef8d45ddd14036b38db84a76ceb413 in term 1.
I20260929 02:56:09.053085 24594 raft_consensus.cc:2454] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62ef8d45ddd14036b38db84a76ceb413 in term 1.
I20260929 02:56:09.053431 24791 leader_election.cc:302] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62ef8d45ddd14036b38db84a76ceb413, 9c43e658a90f4bca833bda8db8a8dcbf; no voters:
I20260929 02:56:09.053686 24925 raft_consensus.cc:2781] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:09.054095 24925 raft_consensus.cc:686] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 LEADER]: Becoming Leader. State: Replica: 62ef8d45ddd14036b38db84a76ceb413, State: Running, Role: LEADER
I20260929 02:56:09.054503 24925 consensus_queue.cc:237] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } }
I20260929 02:56:09.058884 24453 catalog_manager.cc:5495] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3). New cstate: current_term: 1 leader_uuid: "62ef8d45ddd14036b38db84a76ceb413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62ef8d45ddd14036b38db84a76ceb413" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 40683 } health_report { overall_health: HEALTHY } } }
I20260929 02:56:09.150856 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.145933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.8.57.3:40683)
0929 02:56:09.146103 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:09.146107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:09.146124 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:56:09.146257 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:09.146261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:09.146334 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:09.146448 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:56:09.146721 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.147416 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.148757 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.149162 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.149262 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.149801 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:56:09.150297 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:56:09.150421 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:09.150527 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 02:56:09.150595 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:56:09.159498 24594 raft_consensus.cc:1261] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Refusing update from remote peer 62ef8d45ddd14036b38db84a76ceb413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:09.159497 24726 raft_consensus.cc:1261] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Refusing update from remote peer 62ef8d45ddd14036b38db84a76ceb413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:09.160454 24928 consensus_queue.cc:1035] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8188632b7354b4f8c75ee098bb17b6e" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:09.160773 24925 consensus_queue.cc:1035] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c43e658a90f4bca833bda8db8a8dcbf" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 43679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:09.185518 24454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:09.185793 24454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:09.188999 24454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Sending DeleteTablet for 3 replicas of tablet f9e776250fbf40ec937807230076b652
I20260929 02:56:09.190030 24706 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e776250fbf40ec937807230076b652 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:09 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:59888
I20260929 02:56:09.190194 24837 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e776250fbf40ec937807230076b652 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:09 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:57392
I20260929 02:56:09.190479 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 24491
I20260929 02:56:09.190711 24574 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e776250fbf40ec937807230076b652 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:09 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:40184
I20260929 02:56:09.196564 24941 tablet_replica.cc:330] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: stopping tablet replica
I20260929 02:56:09.197057 24941 raft_consensus.cc:2229] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:09.198355 24941 raft_consensus.cc:2258] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:09.204159 24941 ts_tablet_manager.cc:1905] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:09.210543 24942 tablet_replica.cc:330] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: stopping tablet replica
I20260929 02:56:09.211014 24942 raft_consensus.cc:2229] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:56:09.211236 24941 ts_tablet_manager.cc:1918] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:09.211383 24941 log.cc:1196] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/f9e776250fbf40ec937807230076b652
I20260929 02:56:09.211781 24941 ts_tablet_manager.cc:1939] T f9e776250fbf40ec937807230076b652 P 9c43e658a90f4bca833bda8db8a8dcbf: Deleting consensus metadata
I20260929 02:56:09.211764 24942 raft_consensus.cc:2258] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:09.213539 24439 catalog_manager.cc:4841] TS 9c43e658a90f4bca833bda8db8a8dcbf (127.8.57.2:43679): tablet f9e776250fbf40ec937807230076b652 (table test-table [id=14210f506f4a4f6782a023346a7deb30]) successfully deleted
I20260929 02:56:09.201936 24943 tablet_replica.cc:330] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: stopping tablet replica
I20260929 02:56:09.213996 24942 ts_tablet_manager.cc:1905] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:09.214362 24943 raft_consensus.cc:2229] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:09.214977 24943 raft_consensus.cc:2258] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:09.217089 24943 ts_tablet_manager.cc:1905] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:09.220949 24942 ts_tablet_manager.cc:1918] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:09.221172 24942 log.cc:1196] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/f9e776250fbf40ec937807230076b652
I20260929 02:56:09.221728 24942 ts_tablet_manager.cc:1939] T f9e776250fbf40ec937807230076b652 P 62ef8d45ddd14036b38db84a76ceb413: Deleting consensus metadata
I20260929 02:56:09.222512 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.191204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.8.57.1:46703)
0929 02:56:09.191390 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:09.191397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:09.202472 (+ 11075us) server_negotiation.cc:407] Connection header received
0929 02:56:09.202917 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:09.202923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:09.203039 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:09.203115 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:09.205268 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.206399 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.208516 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.209163 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.209264 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.214700 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:09.214718 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:09.214736 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:09.214781 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:09.218846 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:09.219747 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:09.219756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:09.219759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:09.219960 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:09.220434 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:09.220439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:09.220443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:09.220935 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:09.221241 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:09.221868 (+ 627us) server_negotiation.cc:299] Negotiation successful
0929 02:56:09.222128 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:56:09.223407 24441 catalog_manager.cc:4841] TS 62ef8d45ddd14036b38db84a76ceb413 (127.8.57.3:40683): tablet f9e776250fbf40ec937807230076b652 (table test-table [id=14210f506f4a4f6782a023346a7deb30]) successfully deleted
I20260929 02:56:09.223800 24943 ts_tablet_manager.cc:1918] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:09.223987 24943 log.cc:1196] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/f9e776250fbf40ec937807230076b652
I20260929 02:56:09.224315 24634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:09.327190 24943 ts_tablet_manager.cc:1939] T f9e776250fbf40ec937807230076b652 P c8188632b7354b4f8c75ee098bb17b6e: Deleting consensus metadata
I20260929 02:56:09.328022 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 24491
W20260929 02:56:09.341759 24440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.57.1:46703 (error 108)
I20260929 02:56:09.341964 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 24643
I20260929 02:56:09.345748 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.343264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40204)
0929 02:56:09.345421 (+ 2157us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.345544 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2021,"thread_start_us":1937,"threads_started":1}
W20260929 02:56:09.346009 24440 catalog_manager.cc:4584] TS c8188632b7354b4f8c75ee098bb17b6e (127.8.57.1:46703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9e776250fbf40ec937807230076b652: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
I20260929 02:56:09.355406 24914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.342749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.8.57.2:43679)
0929 02:56:09.342930 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:09.342936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:09.343023 (+ 87us) server_negotiation.cc:407] Connection header received
0929 02:56:09.343221 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:09.343226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:09.343312 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:09.343423 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:09.345120 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.345949 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.347232 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.347638 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.347743 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.350168 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:09.350181 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:09.350196 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:09.350229 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:09.352814 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:09.353487 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:09.353494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:09.353497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:09.353623 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:09.353970 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:09.353974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:09.353976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:09.354315 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:09.354580 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:09.354882 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 02:56:09.355094 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:56:09.356211 24766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:09.399641 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.399101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40210)
0929 02:56:09.399356 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.399464 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260929 02:56:09.400897 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.400487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40214)
0929 02:56:09.400659 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.400744 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 02:56:09.443943 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 24643
I20260929 02:56:09.461256 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.460834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40224)
0929 02:56:09.461028 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.461115 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 02:56:09.462275 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.461928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40240)
0929 02:56:09.462069 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.462151 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:56:09.463228 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 24774
I20260929 02:56:09.478107 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.463906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57410 (local address 127.8.57.3:40683)
0929 02:56:09.464094 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:09.464100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:09.464121 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:56:09.464187 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:09.464192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:09.464278 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:09.464421 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:09.465689 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.466839 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.468339 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:09.468997 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:09.469135 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:09.471425 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:09.471440 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:09.471463 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:09.471517 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:09.475216 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:09.475964 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:09.475973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:09.475977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:09.476166 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:09.476496 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:09.476502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:09.476505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:09.476976 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:09.477280 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:09.477493 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 02:56:09.477706 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:56:09.478881 24897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:09.542958 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.542496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40246)
0929 02:56:09.542697 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.542805 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 02:56:09.544078 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.543647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40250)
0929 02:56:09.543835 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.543940 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 02:56:09.598937 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 24774
I20260929 02:56:09.614147 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 24421
I20260929 02:56:09.615247 24484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:09.728123 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:09.727407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46703 (local address 127.0.0.1:40258)
0929 02:56:09.727738 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:09.727918 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:46703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260929 02:56:09.728298 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 24421
I20260929 02:56:29.745489 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41191
--webserver_interface=127.8.57.62
--webserver_port=34399
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:30.053638 24951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:30.053985 24951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:30.054126 24951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:30.061388 24951 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 02:56:30.061470 24951 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 02:56:30.061522 24951 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 02:56:30.061595 24951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:56:30.061654 24951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:30.061753 24951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:30.061805 24951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:56:30.061877 24951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:56:30.071410 24951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=34399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:30.072603 24951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:30.073877 24951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:30.084925 24956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:30.084956 24957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:30.084985 24959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:30.090766 24951 server_base.cc:1034] running on GCE node
I20260929 02:56:30.091455 24951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:30.092751 24951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:30.093950 24951 hybrid_clock.cc:648] HybridClock initialized: now 1790650590093894 us; error 33 us; skew 500 ppm
I20260929 02:56:30.094244 24951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:30 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:30.097640 24951 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:56:30.099596 24951 webserver.cc:460] Webserver started at http://127.8.57.62:34399/ using document root <none> and password file <none>
I20260929 02:56:30.100510 24951 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:30.100680 24951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:30.109594 24951 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 02:56:30.113276 24966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:30.115020 24951 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:56:30.115278 24951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "4e5ea13a287842959dfd3d14eb0b5fab"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "89ae472867c2b3904416bbce2c983e1f"
server_key_iv: "84af357e4d476f5e1b580c51044eaaa2"
server_key_version: "encryptionkey@0"
I20260929 02:56:30.116639 24951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:30.133265 24951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:30.137732 24951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:30.137989 24951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:30.154670 24951 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41191
I20260929 02:56:30.154709 25017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41191 every 8 connection(s)
I20260929 02:56:30.156164 24951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:56:30.162953 25018 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:56:30.165994 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 24951
I20260929 02:56:30.166352 25018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap starting.
I20260929 02:56:30.166998 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:46703
--local_ip_for_outbound_sockets=127.8.57.1
--tserver_master_addrs=127.8.57.62:41191
--webserver_port=44789
--webserver_interface=127.8.57.1
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:56:30.192286 25018 log.cc:824] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:30.208892 25018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:56:30.209388 25018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Bootstrap complete.
I20260929 02:56:30.218551 25018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:30.219391 25018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5ea13a287842959dfd3d14eb0b5fab, State: Initialized, Role: FOLLOWER
I20260929 02:56:30.220009 25018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:30.220218 25018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:56:30.220323 25018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:56:30.220471 25018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 2 FOLLOWER]: Advancing to term 3
I20260929 02:56:30.224179 25018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:30.224618 25018 leader_election.cc:302] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e5ea13a287842959dfd3d14eb0b5fab; no voters:
I20260929 02:56:30.225217 25018 leader_election.cc:288] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 02:56:30.225382 25022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 3 FOLLOWER]: Leader election won for term 3
I20260929 02:56:30.226923 25018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:56:30.230507 25022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [term 3 LEADER]: Becoming Leader. State: Replica: 4e5ea13a287842959dfd3d14eb0b5fab, State: Running, Role: LEADER
I20260929 02:56:30.231053 25022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } }
I20260929 02:56:30.233629 25023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } } }
I20260929 02:56:30.233651 25024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e5ea13a287842959dfd3d14eb0b5fab. Latest consensus state: current_term: 3 leader_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5ea13a287842959dfd3d14eb0b5fab" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41191 } } }
I20260929 02:56:30.233989 25023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:30.234022 25024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:30.248804 25032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:56:30.257088 25032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:56:30.258607 25032 catalog_manager.cc:1259] Loaded cluster ID: d841d6af8e6d477d8c922e9a8c15087c
I20260929 02:56:30.258709 25032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:56:30.262566 25032 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:56:30.264770 25032 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Loaded TSK: 0
I20260929 02:56:30.264901 25032 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e5ea13a287842959dfd3d14eb0b5fab: Loaded TSK: 1
F20260929 02:56:30.265918 25032 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7cc6dc49ad google::LogMessage::Fail() at ??:0
@ 0x7f7cc6dc788a google::LogMessage::SendToLog() at ??:0
@ 0x7f7cc6dc44de google::LogMessage::Flush() at ??:0
@ 0x7f7cc6dc5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7ccab287d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7ccab1773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f7cdc7f31c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f7cdc7f2783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7cdc87e2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f7cdc87e0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f7cdc8c85e9 std::function<>::operator()() at ??:0
@ 0x7f7cdc7f6cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f7cdc7f4c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7cdc87c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f7cdc87c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7cc9a10345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7cc9a1391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7cc9a13799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7cc99eeeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7ccccc86db start_thread at ??:0
@ 0x7f7cc5c5e61f clone at ??:0
W20260929 02:56:30.527429 25020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:30.527720 25020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:30.527846 25020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:30.534961 25020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:30.535079 25020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:30.535225 25020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:56:30.545188 25020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:46703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=44789
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:30.546393 25020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:30.547657 25020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:30.559674 25065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:30.559796 25066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:30.560145 25068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:30.560823 25020 server_base.cc:1034] running on GCE node
I20260929 02:56:30.561372 25020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:30.562273 25020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:30.563457 25020 hybrid_clock.cc:648] HybridClock initialized: now 1790650590563404 us; error 52 us; skew 500 ppm
I20260929 02:56:30.563728 25020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:30 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:30.566763 25020 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:56:30.568219 25020 webserver.cc:460] Webserver started at http://127.8.57.1:44789/ using document root <none> and password file <none>
I20260929 02:56:30.568849 25020 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:30.568945 25020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:30.574018 25020 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:56:30.576581 25075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:30.577893 25020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:30.578022 25020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "c8188632b7354b4f8c75ee098bb17b6e"
format_stamp: "Formatted at 2026-09-29 02:55:44 on dist-test-slave-1dkt"
server_key: "48764635a77c90f580dae7686df3305c"
server_key_iv: "0b3020141bf6ab7ede22cae5bd42713b"
server_key_version: "encryptionkey@0"
I20260929 02:56:30.578938 25020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:30.602774 25020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:30.606349 25020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:30.606561 25020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:30.607538 25020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:30.608803 25020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:30.608886 25020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:30.608960 25020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:30.609019 25020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:30.638283 25020 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:46703
I20260929 02:56:30.638350 25187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:46703 every 8 connection(s)
I20260929 02:56:30.639667 25020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:56:30.644287 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:30.642443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:38079)
0929 02:56:30.643152 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:30.643376 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":279,"threads_started":1}
W20260929 02:56:30.644824 25188 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:41191 (0 consecutive failures): Network error: Failed to ping master at 127.8.57.62:41191: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
I20260929 02:56:30.645500 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25020
I20260929 02:56:30.645733 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:30.645358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:41689)
0929 02:56:30.645512 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:30.645598 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 02:56:30.646265 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:43679
--local_ip_for_outbound_sockets=127.8.57.2
--tserver_master_addrs=127.8.57.62:41191
--webserver_port=33705
--webserver_interface=127.8.57.2
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:56:30.646526 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:30.646198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:38441)
0929 02:56:30.646313 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:30.646395 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 02:56:30.646751 25188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:56:30.953935 25191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:30.954185 25191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:30.954310 25191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:30.961551 25191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:30.961669 25191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:30.961810 25191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:56:30.971311 25191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:43679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=33705
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:30.972477 25191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:30.973711 25191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:30.985610 25199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:30.985677 25197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:30.985652 25196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:30.986446 25191 server_base.cc:1034] running on GCE node
I20260929 02:56:30.986984 25191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:30.987823 25191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:30.989020 25191 hybrid_clock.cc:648] HybridClock initialized: now 1790650590988974 us; error 44 us; skew 500 ppm
I20260929 02:56:30.989283 25191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:30 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650590, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:30.992182 25191 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:56:30.993438 25191 webserver.cc:460] Webserver started at http://127.8.57.2:33705/ using document root <none> and password file <none>
I20260929 02:56:30.994046 25191 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:30.994159 25191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:30.999389 25191 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:56:31.001981 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:31.003252 25191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:56:31.003399 25191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "9c43e658a90f4bca833bda8db8a8dcbf"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "c4065c86f928a53e917bdb87dd56e436"
server_key_iv: "500a7f1036640e8ace4f6716e47e04e6"
server_key_version: "encryptionkey@0"
I20260929 02:56:31.004362 25191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:31.021588 25191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:31.025184 25191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:31.025413 25191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:31.026309 25191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:31.027509 25191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:31.027618 25191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:31.027724 25191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:31.027789 25191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:31.056734 25191 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:43679
I20260929 02:56:31.056782 25318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:43679 every 8 connection(s)
I20260929 02:56:31.058103 25191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:56:31.062635 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.060883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:36707)
0929 02:56:31.061607 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.061790 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":281,"threads_started":1}
W20260929 02:56:31.063241 25319 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:41191 (0 consecutive failures): Network error: Failed to ping master at 127.8.57.62:41191: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
I20260929 02:56:31.063964 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.063631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:53481)
0929 02:56:31.063764 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.063829 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:56:31.064725 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.064406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:34721)
0929 02:56:31.064551 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.064623 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:56:31.064946 25319 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:56:31.065357 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25191
I20260929 02:56:31.065949 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:40683
--local_ip_for_outbound_sockets=127.8.57.3
--tserver_master_addrs=127.8.57.62:41191
--webserver_port=42725
--webserver_interface=127.8.57.3
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:31.371464 25322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:31.371716 25322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:31.371845 25322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:31.379125 25322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:31.379230 25322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:31.379344 25322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:56:31.388947 25322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:40683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=42725
--tserver_master_addrs=127.8.57.62:41191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:31.390077 25322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:31.391345 25322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:31.403532 25327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:31.403486 25330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:31.403591 25328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:31.404266 25322 server_base.cc:1034] running on GCE node
I20260929 02:56:31.404805 25322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:31.405659 25322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:31.406869 25322 hybrid_clock.cc:648] HybridClock initialized: now 1790650591406813 us; error 41 us; skew 500 ppm
I20260929 02:56:31.407202 25322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:31 dist-test-slave-1dkt krb5kdc[23870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650591, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:31.410135 25322 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:56:31.411423 25322 webserver.cc:460] Webserver started at http://127.8.57.3:42725/ using document root <none> and password file <none>
I20260929 02:56:31.412084 25322 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:31.412196 25322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:31.417071 25322 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:56:31.419459 25337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:31.420692 25322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:56:31.420841 25322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "62ef8d45ddd14036b38db84a76ceb413"
format_stamp: "Formatted at 2026-09-29 02:55:45 on dist-test-slave-1dkt"
server_key: "94caff26155b72559dbc400ae999dbc6"
server_key_iv: "0747acebfafaa98fee66966b463d2781"
server_key_version: "encryptionkey@0"
I20260929 02:56:31.421723 25322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:31.433799 25322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:31.437500 25322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:31.437722 25322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:31.438531 25322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:31.439702 25322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:31.439798 25322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:31.439919 25322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:31.439987 25322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:31.467471 25322 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:40683
I20260929 02:56:31.467542 25449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:40683 every 8 connection(s)
I20260929 02:56:31.468834 25322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:56:31.473135 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.471458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:40313)
0929 02:56:31.472151 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.472340 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":259,"threads_started":1}
W20260929 02:56:31.473752 25450 heartbeater.cc:644] Failed to heartbeat to 127.8.57.62:41191 (0 consecutive failures): Network error: Failed to ping master at 127.8.57.62:41191: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
I20260929 02:56:31.474452 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.474162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:55335)
0929 02:56:31.474296 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.474357 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:56:31.475180 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.474892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:38279)
0929 02:56:31.475014 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.475082 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:56:31.475376 25450 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:56:31.475605 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25322
I20260929 02:56:31.648037 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:31.647243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:59961)
0929 02:56:31.647678 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:31.647776 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":198,"threads_started":1}
I20260929 02:56:32.066134 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:32.065394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:60269)
0929 02:56:32.065815 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:32.065899 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":200,"threads_started":1}
I20260929 02:56:32.476683 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:32.475968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:40431)
0929 02:56:32.476385 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:32.476468 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":195,"threads_started":1}
I20260929 02:56:32.649696 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:32.648949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:49803)
0929 02:56:32.649439 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:32.649523 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":245,"threads_started":1}
I20260929 02:56:33.067572 25458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:33.066891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:34901)
0929 02:56:33.067317 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:33.067398 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":211,"threads_started":1}
I20260929 02:56:33.478166 25459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:33.477481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:43543)
0929 02:56:33.477922 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:33.478006 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":212,"threads_started":1}
I20260929 02:56:33.651270 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:33.650528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:39701)
0929 02:56:33.651025 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:33.651101 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":305,"threads_started":1}
I20260929 02:56:34.069093 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:34.068394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:35405)
0929 02:56:34.068840 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:34.068924 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":217,"threads_started":1}
I20260929 02:56:34.479666 25462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:34.478983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:39435)
0929 02:56:34.479408 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:34.479491 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":220,"threads_started":1}
I20260929 02:56:34.652830 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:34.652115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:34559)
0929 02:56:34.652528 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:34.652605 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":204,"threads_started":1}
I20260929 02:56:35.070582 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:35.069915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:48757)
0929 02:56:35.070321 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:35.070398 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":200,"threads_started":1}
I20260929 02:56:35.481257 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:35.480567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:37549)
0929 02:56:35.480983 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:35.481058 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":200,"threads_started":1}
I20260929 02:56:35.654381 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:35.653667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:53947)
0929 02:56:35.654113 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:35.654194 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":218,"threads_started":1}
I20260929 02:56:36.072088 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:36.071390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:55865)
0929 02:56:36.071821 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:36.071923 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":211,"threads_started":1}
I20260929 02:56:36.482923 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:36.482129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:45129)
0929 02:56:36.482666 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:36.482753 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":321,"threads_started":1}
I20260929 02:56:36.656081 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:36.655206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:58065)
0929 02:56:36.655690 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:36.655839 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":260,"threads_started":1}
I20260929 02:56:37.073614 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:37.072892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:51647)
0929 02:56:37.073359 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:37.073452 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":228,"threads_started":1}
I20260929 02:56:37.484483 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:37.483749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:54909)
0929 02:56:37.484231 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:37.484322 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":261,"threads_started":1}
I20260929 02:56:37.657747 25472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:37.656926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:49867)
0929 02:56:37.657421 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:37.657530 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":196,"threads_started":1}
I20260929 02:56:38.075191 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:38.074415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:37487)
0929 02:56:38.074900 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:38.074977 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":235,"threads_started":1}
I20260929 02:56:38.485971 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:38.485267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:46571)
0929 02:56:38.485684 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:38.485763 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":183,"threads_started":1}
I20260929 02:56:38.659339 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:38.658651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:55785)
0929 02:56:38.659074 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:38.659151 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":216,"threads_started":1}
I20260929 02:56:39.076704 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:39.076039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:58647)
0929 02:56:39.076465 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:39.076545 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":210,"threads_started":1}
I20260929 02:56:39.487525 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:39.486826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:48263)
0929 02:56:39.487278 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:39.487364 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260929 02:56:39.660841 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:39.660132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:39143)
0929 02:56:39.660595 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:39.660673 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260929 02:56:40.078228 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:40.077487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:51767)
0929 02:56:40.077986 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:40.078059 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":272,"threads_started":1}
I20260929 02:56:40.489107 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:40.488378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:33747)
0929 02:56:40.488858 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:40.488937 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":232,"threads_started":1}
I20260929 02:56:40.662487 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:40.661755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:48185)
0929 02:56:40.662240 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:40.662325 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":260,"threads_started":1}
I20260929 02:56:41.079787 25482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:41.079073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:41767)
0929 02:56:41.079492 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:41.079578 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":224,"threads_started":1}
I20260929 02:56:41.490759 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:41.490021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:43105)
0929 02:56:41.490469 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:41.490565 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
I20260929 02:56:41.664304 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:41.663488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:32805)
0929 02:56:41.663997 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:41.664089 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":268,"threads_started":1}
I20260929 02:56:42.082332 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:42.080764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:41671)
0929 02:56:42.082040 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:42.082140 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":1022,"threads_started":1}
I20260929 02:56:42.492399 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:42.491659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:49755)
0929 02:56:42.492122 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:42.492221 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
I20260929 02:56:42.665925 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:42.665207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:42633)
0929 02:56:42.665668 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:42.665768 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":235,"threads_started":1}
I20260929 02:56:43.084098 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:43.083317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:37257)
0929 02:56:43.083807 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:43.083931 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":247,"threads_started":1}
I20260929 02:56:43.494014 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:43.493353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:45051)
0929 02:56:43.493770 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:43.493860 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":201,"threads_started":1}
I20260929 02:56:43.667766 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:43.666907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:49783)
0929 02:56:43.667469 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:43.667566 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":291,"threads_started":1}
I20260929 02:56:44.085902 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:44.085132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:44607)
0929 02:56:44.085617 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:44.085704 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":243,"threads_started":1}
I20260929 02:56:44.495535 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:44.494840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:54519)
0929 02:56:44.495260 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:44.495340 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":197,"threads_started":1}
I20260929 02:56:44.669332 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:44.668653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:53799)
0929 02:56:44.669056 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:44.669134 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":202,"threads_started":1}
I20260929 02:56:45.087437 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:45.086770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:46363)
0929 02:56:45.087176 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:45.087251 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":210,"threads_started":1}
I20260929 02:56:45.497073 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:45.496376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:52647)
0929 02:56:45.496819 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:45.496897 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":206,"threads_started":1}
I20260929 02:56:45.671129 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:45.670402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.1:47595)
0929 02:56:45.670853 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:45.670940 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":221,"threads_started":1}
I20260929 02:56:46.089254 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:46.088550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.2:43275)
0929 02:56:46.088981 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:46.089066 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":235,"threads_started":1}
I20260929 02:56:46.484900 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25020
I20260929 02:56:46.498944 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:46.498130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41191 (local address 127.8.57.3:35261)
0929 02:56:46.498611 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:46.498716 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.62:41191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":238,"threads_started":1}
I20260929 02:56:46.505005 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:46.485977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.8.57.1:46703)
0929 02:56:46.486527 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:46.486541 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:46.486578 (+ 37us) server_negotiation.cc:407] Connection header received
0929 02:56:46.488165 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:46.488174 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:46.488605 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:46.488888 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:46.490246 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:46.491452 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:46.492692 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:46.493352 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:46.493479 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:46.499399 (+ 5920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:46.499407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:46.499423 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:46.499460 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:46.502210 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:46.502818 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:46.502824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:46.502830 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:46.502986 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:46.503260 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:46.503278 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:46.503280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:46.503746 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:46.504001 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:46.504401 (+ 400us) server_negotiation.cc:299] Negotiation successful
0929 02:56:46.504730 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":246,"threads_started":1}
I20260929 02:56:46.506459 25183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:46.593963 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25020
I20260929 02:56:46.611829 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25191
I20260929 02:56:46.629725 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:46.612759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.8.57.2:43679)
0929 02:56:46.613423 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:46.613444 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:46.613493 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:56:46.615689 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:46.615704 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:46.616335 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:46.616730 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:46.617902 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:46.619136 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:46.620426 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:46.621108 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:46.621267 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:46.623713 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:46.623755 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:46.623772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:46.623816 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:46.626782 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:46.627435 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:46.627442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:46.627448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:46.627616 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:46.628040 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:46.628044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:46.628046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:46.628540 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:46.628772 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:46.629081 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 02:56:46.629443 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":288,"threads_started":1}
I20260929 02:56:46.631014 25314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:46.712704 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25191
I20260929 02:56:46.726248 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25322
I20260929 02:56:46.743333 25503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:46.727145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.8.57.3:40683)
0929 02:56:46.727700 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:46.727717 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:46.727755 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:56:46.729366 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:46.729375 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:46.729791 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:46.730067 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:46.731417 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:46.732730 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:46.734010 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:46.734701 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:46.734858 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:46.737421 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:46.737429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:46.737446 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:46.737489 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:46.740322 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:46.740962 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:46.740969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:46.740975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:46.741140 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:46.741460 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:46.741463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:46.741465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:46.742103 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:46.742325 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:46.742679 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 02:56:46.743027 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":242,"threads_started":1}
I20260929 02:56:46.744696 25445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:46.826138 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25322
2026-09-29T02:56:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64059 ms)
[----------] 30 tests from SecurityITest (285357 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:56:46 dist-test-slave-1dkt krb5kdc[25507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:56:46 dist-test-slave-1dkt krb5kdc[25507](info): set up 2 sockets
Sep 29 02:56:46 dist-test-slave-1dkt krb5kdc[25507](info): commencing operation
krb5kdc: starting...
W20260929 02:56:47.835731 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:56:47 dist-test-slave-1dkt krb5kdc[25507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:56:48.065505 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36173
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:36173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:48.382938 25520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:48.383230 25520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:48.383361 25520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:48.390522 25520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:56:48.390650 25520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:48.390733 25520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:48.390812 25520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:56:48.390874 25520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:56:48.400727 25520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:36173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:48.401996 25520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:48.403234 25520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:48.414618 25526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:48.414644 25528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:48.414670 25525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:48.415495 25520 server_base.cc:1034] running on GCE node
I20260929 02:56:48.416313 25520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:48.417646 25520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:48.418843 25520 hybrid_clock.cc:648] HybridClock initialized: now 1790650608418799 us; error 32 us; skew 500 ppm
I20260929 02:56:48.419145 25520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:48 dist-test-slave-1dkt krb5kdc[25507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:48.422101 25520 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:56:48.423460 25520 webserver.cc:460] Webserver started at http://127.8.57.62:36863/ using document root <none> and password file <none>
I20260929 02:56:48.424149 25520 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:48.424268 25520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:48.424539 25520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:48.427410 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "8355e321b830467fb1e153858bd27d4b"
format_stamp: "Formatted at 2026-09-29 02:56:48 on dist-test-slave-1dkt"
server_key: "e53a19cae510920087a118dd27413ca0"
server_key_iv: "60c367b3ab2df7be886c84110908cddb"
server_key_version: "encryptionkey@0"
I20260929 02:56:48.428171 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "8355e321b830467fb1e153858bd27d4b"
format_stamp: "Formatted at 2026-09-29 02:56:48 on dist-test-slave-1dkt"
server_key: "e53a19cae510920087a118dd27413ca0"
server_key_iv: "60c367b3ab2df7be886c84110908cddb"
server_key_version: "encryptionkey@0"
I20260929 02:56:48.433070 25520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:56:48.436386 25535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:48.437654 25520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:56:48.437824 25520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "8355e321b830467fb1e153858bd27d4b"
format_stamp: "Formatted at 2026-09-29 02:56:48 on dist-test-slave-1dkt"
server_key: "e53a19cae510920087a118dd27413ca0"
server_key_iv: "60c367b3ab2df7be886c84110908cddb"
server_key_version: "encryptionkey@0"
I20260929 02:56:48.438005 25520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:48.475606 25520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:48.479321 25520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:48.479573 25520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:48.496169 25520 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:36173
I20260929 02:56:48.496197 25586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:36173 every 8 connection(s)
I20260929 02:56:48.497435 25520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:56:48.500932 25587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:48.506373 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25520
I20260929 02:56:48.506568 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:56:48.507215 8420 external_mini_cluster.cc:1428] Setting key cf1033e0cf3ab82aad8b32f70d6b168a
I20260929 02:56:48.511168 25587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b: Bootstrap starting.
I20260929 02:56:48.514608 25587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:48.515594 25587 log.cc:824] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:48.518443 25587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b: No bootstrap required, opened a new log
Sep 29 02:56:48 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:48.527281 25587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355e321b830467fb1e153858bd27d4b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36173 } }
I20260929 02:56:48.527667 25587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:48.527824 25587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8355e321b830467fb1e153858bd27d4b, State: Initialized, Role: FOLLOWER
I20260929 02:56:48.528426 25587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355e321b830467fb1e153858bd27d4b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36173 } }
I20260929 02:56:48.528645 25587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:56:48.528774 25587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:56:48.528997 25587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:48.529975 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:48.509078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.8.57.62:36173)
0929 02:56:48.509784 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:48.509795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:48.509834 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:56:48.511494 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:48.511515 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:48.511919 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:48.512223 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:48.514060 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:48.515435 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:48.517102 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:48.517813 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:48.517946 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:48.522730 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:48.522738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:48.522752 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:48.522793 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:48.525948 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:48.526739 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:48.526745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:48.526750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:48.526903 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:48.527272 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:48.527275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:48.527276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:48.527898 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:48.528186 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:48.528591 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 02:56:48.528995 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":329,"threads_started":1}
I20260929 02:56:48.531452 25587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355e321b830467fb1e153858bd27d4b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36173 } }
I20260929 02:56:48.532014 25587 leader_election.cc:302] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8355e321b830467fb1e153858bd27d4b; no voters:
I20260929 02:56:48.532653 25587 leader_election.cc:288] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:56:48.532786 25592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:48.533422 25592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [term 1 LEADER]: Becoming Leader. State: Replica: 8355e321b830467fb1e153858bd27d4b, State: Running, Role: LEADER
I20260929 02:56:48.533898 25592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355e321b830467fb1e153858bd27d4b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36173 } }
I20260929 02:56:48.534528 25587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:56:48.536614 25594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8355e321b830467fb1e153858bd27d4b. Latest consensus state: current_term: 1 leader_uuid: "8355e321b830467fb1e153858bd27d4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355e321b830467fb1e153858bd27d4b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36173 } } }
I20260929 02:56:48.536803 25593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8355e321b830467fb1e153858bd27d4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355e321b830467fb1e153858bd27d4b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36173 } } }
I20260929 02:56:48.537046 25594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:48.537050 25593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:48.539163 25600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:56:48.544320 25600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:56:48.552686 25600 catalog_manager.cc:1347] Generated new cluster ID: 07aba674200d4a29aa61d84f3686938c
I20260929 02:56:48.552786 25600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:56:48.572006 25600 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:56:48.573697 25600 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:56:48.581763 25600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b: Generated new TSK 0
I20260929 02:56:48.582691 25600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:56:48.645737 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36173
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:48.956518 25613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:48.956799 25613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:48.956889 25613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:48.964031 25613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:48.964133 25613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:48.964246 25613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:56:48.973843 25613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:48.975004 25613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:48.976243 25613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:48.988559 25618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:48.988523 25621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:48.988588 25619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:48.989250 25613 server_base.cc:1034] running on GCE node
I20260929 02:56:48.989812 25613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:48.990697 25613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:48.991932 25613 hybrid_clock.cc:648] HybridClock initialized: now 1790650608991878 us; error 44 us; skew 500 ppm
I20260929 02:56:48.992226 25613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:48 dist-test-slave-1dkt krb5kdc[25507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:48.995165 25613 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:56:48.996601 25613 webserver.cc:460] Webserver started at http://127.8.57.1:44691/ using document root <none> and password file <none>
I20260929 02:56:48.997241 25613 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:48.997351 25613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:48.997637 25613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:49.000432 25613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "aa748292d8664b1fab9118a436758c8f"
format_stamp: "Formatted at 2026-09-29 02:56:48 on dist-test-slave-1dkt"
server_key: "c06074df0fc137947aedafcae10e287f"
server_key_iv: "9b9a2173a6cac567e9d66de096746c64"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.001369 25613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "aa748292d8664b1fab9118a436758c8f"
format_stamp: "Formatted at 2026-09-29 02:56:48 on dist-test-slave-1dkt"
server_key: "c06074df0fc137947aedafcae10e287f"
server_key_iv: "9b9a2173a6cac567e9d66de096746c64"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.006958 25613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:56:49.010366 25628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.011653 25613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:56:49.011814 25613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "aa748292d8664b1fab9118a436758c8f"
format_stamp: "Formatted at 2026-09-29 02:56:48 on dist-test-slave-1dkt"
server_key: "c06074df0fc137947aedafcae10e287f"
server_key_iv: "9b9a2173a6cac567e9d66de096746c64"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.012060 25613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:49.033193 25613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:49.037062 25613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:49.037318 25613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:49.038264 25613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:49.039546 25613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:49.039635 25613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.039749 25613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:49.039800 25613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.068898 25613 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:43129
I20260929 02:56:49.068953 25740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:43129 every 8 connection(s)
I20260929 02:56:49.070078 25613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:56:49.076092 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25613
I20260929 02:56:49.076263 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:56:49.076927 8420 external_mini_cluster.cc:1428] Setting key ea4a5ef525eb1dbe50c785e0cb240255
Sep 29 02:56:49 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:49.089190 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:49.072401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:45829 (local address 127.8.57.62:36173)
0929 02:56:49.072819 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:49.072823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:49.073771 (+ 948us) server_negotiation.cc:407] Connection header received
0929 02:56:49.075578 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:49.075583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:49.075673 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:49.075786 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:49.077367 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.078223 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.079738 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.080170 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.080280 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:49.083824 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:49.083829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:49.083832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:49.083863 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:49.086339 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:49.087002 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:49.087007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:49.087012 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:49.087128 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:49.087527 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:49.087532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:49.087536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:49.087817 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:49.087967 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:49.088608 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 02:56:49.088833 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":205,"threads_started":1}
I20260929 02:56:49.090075 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:49.072930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36173 (local address 127.8.57.1:45829)
0929 02:56:49.073614 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:49.073649 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:49.075322 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:49.076056 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:49.076065 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:49.076608 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:49.077184 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:49.077205 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.078391 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.078395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:49.079589 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:49.079602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.080385 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.080392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:49.080610 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:49.081223 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:49.081250 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:49.083609 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:49.086476 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:49.086484 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:49.086500 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:49.086895 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:49.087232 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:49.087237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:49.087241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:49.087427 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:49.087959 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:49.087967 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:49.088274 (+ 307us) client_negotiation.cc:769] Sending connection context
0929 02:56:49.088511 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 02:56:49.088993 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":280,"threads_started":1}
I20260929 02:56:49.091598 25741 heartbeater.cc:347] Connected to a master server at 127.8.57.62:36173
I20260929 02:56:49.091946 25741 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:49.092720 25741 heartbeater.cc:511] Master 127.8.57.62:36173 requested a full tablet report, sending...
I20260929 02:56:49.094936 25552 ts_manager.cc:194] Registered new tserver with Master: aa748292d8664b1fab9118a436758c8f (127.8.57.1:43129)
I20260929 02:56:49.096365 25552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:45829
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:56:49.126832 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36173
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:49.440351 25747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:49.440593 25747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:49.440685 25747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:49.448135 25747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:49.448235 25747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:49.448348 25747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:56:49.457531 25747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:49.458617 25747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:49.459800 25747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:49.471524 25752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:49.471589 25753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:49.471637 25755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:49.472404 25747 server_base.cc:1034] running on GCE node
I20260929 02:56:49.472934 25747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:49.473757 25747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:49.474967 25747 hybrid_clock.cc:648] HybridClock initialized: now 1790650609474909 us; error 45 us; skew 500 ppm
I20260929 02:56:49.475272 25747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:49 dist-test-slave-1dkt krb5kdc[25507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:49.478185 25747 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:56:49.479475 25747 webserver.cc:460] Webserver started at http://127.8.57.2:40955/ using document root <none> and password file <none>
I20260929 02:56:49.480120 25747 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:49.480222 25747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:49.480463 25747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:49.483253 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "bb155a1fdd694372aa0397f1590fa207"
format_stamp: "Formatted at 2026-09-29 02:56:49 on dist-test-slave-1dkt"
server_key: "21b593849670ac8b3fc1a742beefe40e"
server_key_iv: "37bb66f22d1b439e92ed2d0c7317c9ae"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.484074 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "bb155a1fdd694372aa0397f1590fa207"
format_stamp: "Formatted at 2026-09-29 02:56:49 on dist-test-slave-1dkt"
server_key: "21b593849670ac8b3fc1a742beefe40e"
server_key_iv: "37bb66f22d1b439e92ed2d0c7317c9ae"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.488844 25747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:56:49.491921 25762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.493093 25747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 02:56:49.493211 25747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "bb155a1fdd694372aa0397f1590fa207"
format_stamp: "Formatted at 2026-09-29 02:56:49 on dist-test-slave-1dkt"
server_key: "21b593849670ac8b3fc1a742beefe40e"
server_key_iv: "37bb66f22d1b439e92ed2d0c7317c9ae"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.493360 25747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:49.518610 25747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:49.522621 25747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:49.522869 25747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:49.523772 25747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:49.525064 25747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:49.525133 25747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.525246 25747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:49.525295 25747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.552945 25747 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:38089
I20260929 02:56:49.553009 25874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:38089 every 8 connection(s)
I20260929 02:56:49.554076 25747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:56:49.558945 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25747
I20260929 02:56:49.559104 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:56:49.559506 8420 external_mini_cluster.cc:1428] Setting key 0b9fb9aebc5a86a115eb8d6894c5ce24
Sep 29 02:56:49 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:49.572448 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:49.556375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:53481 (local address 127.8.57.62:36173)
0929 02:56:49.556537 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:49.556541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:49.557623 (+ 1082us) server_negotiation.cc:407] Connection header received
0929 02:56:49.559435 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:49.559440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:49.559526 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:49.559611 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:49.561254 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.561975 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.563427 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.563808 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.563927 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:49.567072 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:49.567078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:49.567081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:49.567113 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:49.569569 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:49.570414 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:49.570418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:49.570420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:49.570556 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:49.570954 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:49.570957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:49.570959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:49.571199 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:49.571307 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:49.572048 (+ 741us) server_negotiation.cc:299] Negotiation successful
0929 02:56:49.572214 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:56:49.573683 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:49.556822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36173 (local address 127.8.57.2:53481)
0929 02:56:49.557476 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:49.557517 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:49.559182 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:49.559844 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:49.559851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:49.560376 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:49.561101 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:49.561126 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.562110 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.562116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:49.563299 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:49.563311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:49.564020 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:49.564027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:49.564233 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:49.564820 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:49.564842 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:49.566872 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:49.569696 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:49.569704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:49.569718 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:49.570307 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:49.570645 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:49.570649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:49.570652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:49.570860 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:49.571309 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:49.571316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:49.571670 (+ 354us) client_negotiation.cc:769] Sending connection context
0929 02:56:49.571971 (+ 301us) client_negotiation.cc:240] Negotiation successful
0929 02:56:49.572456 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":260,"threads_started":1}
I20260929 02:56:49.574991 25875 heartbeater.cc:347] Connected to a master server at 127.8.57.62:36173
I20260929 02:56:49.575277 25875 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:49.576050 25875 heartbeater.cc:511] Master 127.8.57.62:36173 requested a full tablet report, sending...
I20260929 02:56:49.577510 25552 ts_manager.cc:194] Registered new tserver with Master: bb155a1fdd694372aa0397f1590fa207 (127.8.57.2:38089)
I20260929 02:56:49.578544 25552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:53481
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:56:49.610332 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36173
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:49.914698 25880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:49.914956 25880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:49.915047 25880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:49.921999 25880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:49.922093 25880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:49.922199 25880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:56:49.931490 25880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:36173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:49.932626 25880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:49.933836 25880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:49.945935 25885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:49.945971 25886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:49.946007 25888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:49.946698 25880 server_base.cc:1034] running on GCE node
I20260929 02:56:49.947247 25880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:49.948169 25880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:49.949357 25880 hybrid_clock.cc:648] HybridClock initialized: now 1790650609949312 us; error 32 us; skew 500 ppm
I20260929 02:56:49.949615 25880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:49 dist-test-slave-1dkt krb5kdc[25507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:49.952488 25880 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:56:49.953820 25880 webserver.cc:460] Webserver started at http://127.8.57.3:43723/ using document root <none> and password file <none>
I20260929 02:56:49.954463 25880 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:49.954563 25880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:49.954842 25880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:49.957604 25880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "8b23e068e44c4061a83014c0108d07ce"
format_stamp: "Formatted at 2026-09-29 02:56:49 on dist-test-slave-1dkt"
server_key: "0fd94b11cb8cd6ec8d7d90c743396ad4"
server_key_iv: "7a44a19e2898854e9cde2d65fe4c4815"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.958344 25880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "8b23e068e44c4061a83014c0108d07ce"
format_stamp: "Formatted at 2026-09-29 02:56:49 on dist-test-slave-1dkt"
server_key: "0fd94b11cb8cd6ec8d7d90c743396ad4"
server_key_iv: "7a44a19e2898854e9cde2d65fe4c4815"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.963030 25880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:56:49.966195 25895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:49.967392 25880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:56:49.967525 25880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "8b23e068e44c4061a83014c0108d07ce"
format_stamp: "Formatted at 2026-09-29 02:56:49 on dist-test-slave-1dkt"
server_key: "0fd94b11cb8cd6ec8d7d90c743396ad4"
server_key_iv: "7a44a19e2898854e9cde2d65fe4c4815"
server_key_version: "encryptionkey@0"
I20260929 02:56:49.967684 25880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:49.996080 25880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:50.000036 25880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:50.000262 25880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:50.001104 25880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:50.002300 25880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:50.002369 25880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:50.002446 25880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:50.002488 25880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:50.030439 25880 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:42493
I20260929 02:56:50.030510 26007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:42493 every 8 connection(s)
I20260929 02:56:50.031638 25880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:56:50.040400 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 25880
I20260929 02:56:50.040561 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:56:50.040994 8420 external_mini_cluster.cc:1428] Setting key 25f3613be1a6fcc6a757baed691340fe
Sep 29 02:56:50 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:50.050611 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.034138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:53975 (local address 127.8.57.62:36173)
0929 02:56:50.034332 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.034336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.035455 (+ 1119us) server_negotiation.cc:407] Connection header received
0929 02:56:50.037351 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.037358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.037436 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.037521 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:50.039037 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.039797 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.041440 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.041955 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.042114 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.045627 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:50.045633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:50.045636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:50.045672 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:50.048100 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.048773 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.048777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.048780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.048896 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.049216 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.049219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.049222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.049482 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:50.049590 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.050207 (+ 617us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.050376 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:56:50.051474 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.034573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36173 (local address 127.8.57.3:53975)
0929 02:56:50.035312 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.035358 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:50.037060 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:50.037780 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:50.037788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:50.038328 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:50.038872 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.038889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.039964 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.039968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.041311 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.041324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.042196 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.042205 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.042498 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.043155 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:50.043183 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:50.045408 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:50.048226 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:50.048232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:50.048244 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:50.048630 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:50.048973 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:50.048976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:50.048978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:50.049138 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:50.049569 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:50.049574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:50.049853 (+ 279us) client_negotiation.cc:769] Sending connection context
0929 02:56:50.050054 (+ 201us) client_negotiation.cc:240] Negotiation successful
0929 02:56:50.050427 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":265,"threads_started":1}
I20260929 02:56:50.052763 26008 heartbeater.cc:347] Connected to a master server at 127.8.57.62:36173
I20260929 02:56:50.053028 26008 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:50.053653 26008 heartbeater.cc:511] Master 127.8.57.62:36173 requested a full tablet report, sending...
I20260929 02:56:50.054991 25552 ts_manager.cc:194] Registered new tserver with Master: 8b23e068e44c4061a83014c0108d07ce (127.8.57.3:42493)
I20260929 02:56:50.056077 25552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:53975
I20260929 02:56:50.056187 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:56:50.073123 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.060448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.8.57.62:36173)
0929 02:56:50.060590 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.060594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.061044 (+ 450us) server_negotiation.cc:407] Connection header received
0929 02:56:50.061202 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.061205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.061288 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.061368 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:50.062724 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.063421 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.064790 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.065186 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.065290 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.067383 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:50.067389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:50.067392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:50.067422 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:50.070439 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.071140 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.071145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.071148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.071288 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.071645 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.071650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.071654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.072097 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:50.072346 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.072614 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.072836 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:56:50.076993 25552 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:56:50.083446 25552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:56:50.099529 25741 heartbeater.cc:503] Master 127.8.57.62:36173 was elected leader, sending a full tablet report...
I20260929 02:56:50.103591 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.095172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.8.57.3:42493)
0929 02:56:50.096319 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.096327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.096358 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:56:50.096454 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.096460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.096661 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.097578 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:50.097833 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.098913 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.101320 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.102181 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.102291 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.102504 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.103109 (+ 605us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.103263 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":813,"threads_started":1}
I20260929 02:56:50.104091 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.094622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:42493 (local address 127.0.0.1:42764)
0929 02:56:50.095510 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.095546 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:50.095718 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:50.096878 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:50.096885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:50.096916 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:50.097390 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.097404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.099032 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.099037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.101174 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.101186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.102378 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.102384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.102574 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.102624 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 02:56:50.102753 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 02:56:50.102929 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":24,"spinlock_wait_cycles":2944,"thread_start_us":789,"threads_started":1}
I20260929 02:56:50.106616 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.096034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53942)
0929 02:56:50.097627 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.097640 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:50.097762 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:50.098535 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:50.098540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:50.098563 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:50.098904 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.098917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.100800 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.100805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.104066 (+ 3261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.104081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.105845 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.105851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.106008 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.106040 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:56:50.106130 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:56:50.106243 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":176,"thread_start_us":1183,"threads_started":1}
I20260929 02:56:50.106650 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.096551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.8.57.1:43129)
0929 02:56:50.097509 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.097517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.097734 (+ 217us) server_negotiation.cc:407] Connection header received
0929 02:56:50.097905 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.097912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.098146 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.098322 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:50.099080 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.100642 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.104243 (+ 3601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.105649 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.105787 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.106005 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.106189 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.106340 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":796,"threads_started":1}
I20260929 02:56:50.107556 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.094455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:38089 (local address 127.0.0.1:50518)
0929 02:56:50.095513 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.095545 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:50.095758 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:50.096750 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:50.096757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:50.096784 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:50.097791 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.097804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.100503 (+ 2699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.100508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.104063 (+ 3555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.104081 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.106318 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.106324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.106483 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.106517 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:56:50.107075 (+ 558us) client_negotiation.cc:240] Negotiation successful
0929 02:56:50.107196 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":594,"spinlock_wait_cycles":20864,"thread_start_us":810,"threads_started":1}
I20260929 02:56:50.107623 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.094491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.8.57.2:38089)
0929 02:56:50.095108 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.095118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.095915 (+ 797us) server_negotiation.cc:407] Connection header received
0929 02:56:50.096022 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.096029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.096282 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.096457 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:50.097970 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.100362 (+ 2392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.104515 (+ 4153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.105865 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.105954 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.106128 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.107146 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.107317 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":319,"threads_started":1}
I20260929 02:56:50.107653 25943 tablet_service.cc:1467] Processing CreateTablet for tablet c91ee7185ac74b198e3f12fb7bcbb1d2 (DEFAULT_TABLE table=test-table [id=a6d0280cf7eb4a08bb7e6d6e2ca7a5f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:50.109685 25943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c91ee7185ac74b198e3f12fb7bcbb1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:50.109715 25676 tablet_service.cc:1467] Processing CreateTablet for tablet c91ee7185ac74b198e3f12fb7bcbb1d2 (DEFAULT_TABLE table=test-table [id=a6d0280cf7eb4a08bb7e6d6e2ca7a5f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:50.110797 25676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c91ee7185ac74b198e3f12fb7bcbb1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:50.110819 25810 tablet_service.cc:1467] Processing CreateTablet for tablet c91ee7185ac74b198e3f12fb7bcbb1d2 (DEFAULT_TABLE table=test-table [id=a6d0280cf7eb4a08bb7e6d6e2ca7a5f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:50.112057 25810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c91ee7185ac74b198e3f12fb7bcbb1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:50.121345 26026 tablet_bootstrap.cc:492] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Bootstrap starting.
I20260929 02:56:50.123234 26027 tablet_bootstrap.cc:492] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Bootstrap starting.
I20260929 02:56:50.124807 26026 tablet_bootstrap.cc:654] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:50.124861 26028 tablet_bootstrap.cc:492] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Bootstrap starting.
I20260929 02:56:50.126633 26027 tablet_bootstrap.cc:654] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:50.126694 26026 log.cc:824] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:50.127604 26028 tablet_bootstrap.cc:654] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:50.127786 26027 log.cc:824] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:50.128708 26028 log.cc:824] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:50.129598 26026 tablet_bootstrap.cc:492] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: No bootstrap required, opened a new log
I20260929 02:56:50.129854 26026 ts_tablet_manager.cc:1397] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:56:50.130685 26027 tablet_bootstrap.cc:492] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: No bootstrap required, opened a new log
I20260929 02:56:50.130992 26027 ts_tablet_manager.cc:1397] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260929 02:56:50.131232 26028 tablet_bootstrap.cc:492] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: No bootstrap required, opened a new log
I20260929 02:56:50.131498 26028 ts_tablet_manager.cc:1397] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 02:56:50.141891 26028 raft_consensus.cc:348] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.142467 26028 raft_consensus.cc:374] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:50.142609 26028 raft_consensus.cc:729] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb155a1fdd694372aa0397f1590fa207, State: Initialized, Role: FOLLOWER
I20260929 02:56:50.143400 26028 consensus_queue.cc:260] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.143427 26027 raft_consensus.cc:348] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.143926 26027 raft_consensus.cc:374] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:50.144069 26027 raft_consensus.cc:729] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b23e068e44c4061a83014c0108d07ce, State: Initialized, Role: FOLLOWER
I20260929 02:56:50.144591 25875 heartbeater.cc:503] Master 127.8.57.62:36173 was elected leader, sending a full tablet report...
I20260929 02:56:50.144819 26027 consensus_queue.cc:260] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.146828 26028 ts_tablet_manager.cc:1428] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Time spent starting tablet: real 0.015s user 0.011s sys 0.001s
I20260929 02:56:50.145922 26026 raft_consensus.cc:348] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.149194 26026 raft_consensus.cc:374] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:50.149293 26026 raft_consensus.cc:729] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa748292d8664b1fab9118a436758c8f, State: Initialized, Role: FOLLOWER
I20260929 02:56:50.149982 26026 consensus_queue.cc:260] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.151777 26026 ts_tablet_manager.cc:1428] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Time spent starting tablet: real 0.022s user 0.015s sys 0.000s
I20260929 02:56:50.152858 26008 heartbeater.cc:503] Master 127.8.57.62:36173 was elected leader, sending a full tablet report...
I20260929 02:56:50.153409 26027 ts_tablet_manager.cc:1428] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Time spent starting tablet: real 0.022s user 0.011s sys 0.008s
W20260929 02:56:50.283232 26009 tablet.cc:2369] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:56:50.305868 25876 tablet.cc:2369] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:50.321066 26032 raft_consensus.cc:482] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:56:50.321346 26032 raft_consensus.cc:504] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
W20260929 02:56:50.322222 25742 tablet.cc:2369] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:50.323240 26032 leader_election.cc:288] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b23e068e44c4061a83014c0108d07ce (127.8.57.3:42493), aa748292d8664b1fab9118a436758c8f (127.8.57.1:43129)
I20260929 02:56:50.329669 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.323557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:35101 (local address 127.8.57.3:42493)
0929 02:56:50.323836 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.323843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.324572 (+ 729us) server_negotiation.cc:407] Connection header received
0929 02:56:50.324761 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.324765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.324824 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.324899 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:50.325438 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.326161 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.327994 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.328754 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.328846 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.328939 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.329318 (+ 379us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.329439 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 02:56:50.329676 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.323578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:47175 (local address 127.8.57.1:43129)
0929 02:56:50.324026 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.324031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.324437 (+ 406us) server_negotiation.cc:407] Connection header received
0929 02:56:50.324748 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.324754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.324822 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.324898 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:50.325555 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.326322 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.328055 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.328801 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.328926 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.329019 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.329325 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.329453 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260929 02:56:50.329846 26036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.323776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:42493 (local address 127.8.57.2:35101)
0929 02:56:50.324484 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.324498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:50.324617 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:50.324926 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:50.324930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:50.324961 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:50.325295 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.325304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.326285 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.326290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.327830 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.327844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.328975 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.328981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.329128 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.329195 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 02:56:50.329274 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 02:56:50.329406 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":69,"thread_start_us":422,"threads_started":1}
I20260929 02:56:50.329883 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.323512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.8.57.2:47175)
0929 02:56:50.324309 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.324332 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:50.324494 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:50.324929 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:50.324933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:50.325018 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:50.325417 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.325428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.326463 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.326466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.327909 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:50.327923 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.329027 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.329032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:50.329176 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.329198 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:56:50.329275 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:56:50.329408 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":7,"spinlock_wait_cycles":19328,"thread_start_us":392,"threads_started":1}
I20260929 02:56:50.330783 25696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c91ee7185ac74b198e3f12fb7bcbb1d2" candidate_uuid: "bb155a1fdd694372aa0397f1590fa207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa748292d8664b1fab9118a436758c8f" is_pre_election: true
I20260929 02:56:50.330783 25963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c91ee7185ac74b198e3f12fb7bcbb1d2" candidate_uuid: "bb155a1fdd694372aa0397f1590fa207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b23e068e44c4061a83014c0108d07ce" is_pre_election: true
I20260929 02:56:50.331317 25696 raft_consensus.cc:2454] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb155a1fdd694372aa0397f1590fa207 in term 0.
I20260929 02:56:50.331306 25963 raft_consensus.cc:2454] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb155a1fdd694372aa0397f1590fa207 in term 0.
I20260929 02:56:50.332024 25765 leader_election.cc:302] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa748292d8664b1fab9118a436758c8f, bb155a1fdd694372aa0397f1590fa207; no voters:
I20260929 02:56:50.332401 26032 raft_consensus.cc:2781] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:56:50.332557 26032 raft_consensus.cc:482] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:56:50.332657 26032 raft_consensus.cc:3037] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:50.334828 26032 raft_consensus.cc:504] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.335317 26032 leader_election.cc:288] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [CANDIDATE]: Term 1 election: Requested vote from peers 8b23e068e44c4061a83014c0108d07ce (127.8.57.3:42493), aa748292d8664b1fab9118a436758c8f (127.8.57.1:43129)
I20260929 02:56:50.335731 25963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c91ee7185ac74b198e3f12fb7bcbb1d2" candidate_uuid: "bb155a1fdd694372aa0397f1590fa207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b23e068e44c4061a83014c0108d07ce"
I20260929 02:56:50.335757 25696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c91ee7185ac74b198e3f12fb7bcbb1d2" candidate_uuid: "bb155a1fdd694372aa0397f1590fa207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa748292d8664b1fab9118a436758c8f"
I20260929 02:56:50.335932 25963 raft_consensus.cc:3037] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:50.335954 25696 raft_consensus.cc:3037] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:50.338008 25963 raft_consensus.cc:2454] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb155a1fdd694372aa0397f1590fa207 in term 1.
I20260929 02:56:50.338032 25696 raft_consensus.cc:2454] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb155a1fdd694372aa0397f1590fa207 in term 1.
I20260929 02:56:50.338367 25763 leader_election.cc:302] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b23e068e44c4061a83014c0108d07ce, bb155a1fdd694372aa0397f1590fa207; no voters:
I20260929 02:56:50.338640 26032 raft_consensus.cc:2781] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:50.339063 26032 raft_consensus.cc:686] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 1 LEADER]: Becoming Leader. State: Replica: bb155a1fdd694372aa0397f1590fa207, State: Running, Role: LEADER
I20260929 02:56:50.339481 26032 consensus_queue.cc:237] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } }
I20260929 02:56:50.343924 25551 catalog_manager.cc:5495] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb155a1fdd694372aa0397f1590fa207 (127.8.57.2). New cstate: current_term: 1 leader_uuid: "bb155a1fdd694372aa0397f1590fa207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb155a1fdd694372aa0397f1590fa207" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 38089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 } health_report { overall_health: UNKNOWN } } }
I20260929 02:56:50.400898 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.396091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50532 (local address 127.8.57.2:38089)
0929 02:56:50.396272 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.396277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.396295 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:56:50.396465 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.396469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.396545 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.396661 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:56:50.396982 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.397719 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.399072 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.399458 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.399544 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.399952 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:56:50.400418 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:56:50.400538 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.400574 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.400632 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:56:50.408434 25696 raft_consensus.cc:1261] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 1 FOLLOWER]: Refusing update from remote peer bb155a1fdd694372aa0397f1590fa207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:50.408430 25963 raft_consensus.cc:1261] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 1 FOLLOWER]: Refusing update from remote peer bb155a1fdd694372aa0397f1590fa207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:50.409302 26037 consensus_queue.cc:1035] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b23e068e44c4061a83014c0108d07ce" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:50.409600 26032 consensus_queue.cc:1035] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa748292d8664b1fab9118a436758c8f" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:50.431475 25551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:50.431696 25551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:50.434906 25551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8355e321b830467fb1e153858bd27d4b: Sending DeleteTablet for 3 replicas of tablet c91ee7185ac74b198e3f12fb7bcbb1d2
I20260929 02:56:50.435896 25810 tablet_service.cc:1514] Processing DeleteTablet for tablet c91ee7185ac74b198e3f12fb7bcbb1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:50 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:50518
I20260929 02:56:50.435988 25943 tablet_service.cc:1514] Processing DeleteTablet for tablet c91ee7185ac74b198e3f12fb7bcbb1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:50 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:42764
I20260929 02:56:50.436892 25676 tablet_service.cc:1514] Processing DeleteTablet for tablet c91ee7185ac74b198e3f12fb7bcbb1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:50 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:53942
I20260929 02:56:50.437206 26051 tablet_replica.cc:330] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: stopping tablet replica
I20260929 02:56:50.437218 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25613
I20260929 02:56:50.437671 26051 raft_consensus.cc:2229] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:50.438144 26051 raft_consensus.cc:2258] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:50.438910 26050 tablet_replica.cc:330] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: stopping tablet replica
I20260929 02:56:50.439275 26050 raft_consensus.cc:2229] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:56:50.439792 26050 raft_consensus.cc:2258] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:50.441860 26050 ts_tablet_manager.cc:1905] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:50.445040 26051 ts_tablet_manager.cc:1905] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:50.448498 26052 tablet_replica.cc:330] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: stopping tablet replica
Sep 29 02:56:50 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:56:50.449198 26050 ts_tablet_manager.cc:1918] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:50.449441 26050 log.cc:1196] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/c91ee7185ac74b198e3f12fb7bcbb1d2
I20260929 02:56:50.450048 26050 ts_tablet_manager.cc:1939] T c91ee7185ac74b198e3f12fb7bcbb1d2 P bb155a1fdd694372aa0397f1590fa207: Deleting consensus metadata
I20260929 02:56:50.451974 25538 catalog_manager.cc:4841] TS bb155a1fdd694372aa0397f1590fa207 (127.8.57.2:38089): tablet c91ee7185ac74b198e3f12fb7bcbb1d2 (table test-table [id=a6d0280cf7eb4a08bb7e6d6e2ca7a5f0]) successfully deleted
I20260929 02:56:50.452698 26051 ts_tablet_manager.cc:1918] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:50.452677 26052 raft_consensus.cc:2229] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:50.452936 26051 log.cc:1196] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/c91ee7185ac74b198e3f12fb7bcbb1d2
I20260929 02:56:50.453156 26052 raft_consensus.cc:2258] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:50.453678 26051 ts_tablet_manager.cc:1939] T c91ee7185ac74b198e3f12fb7bcbb1d2 P 8b23e068e44c4061a83014c0108d07ce: Deleting consensus metadata
I20260929 02:56:50.455273 25536 catalog_manager.cc:4841] TS 8b23e068e44c4061a83014c0108d07ce (127.8.57.3:42493): tablet c91ee7185ac74b198e3f12fb7bcbb1d2 (table test-table [id=a6d0280cf7eb4a08bb7e6d6e2ca7a5f0]) successfully deleted
I20260929 02:56:50.455325 26052 ts_tablet_manager.cc:1905] T c91ee7185ac74b198e3f12fb7bcbb1d2 P aa748292d8664b1fab9118a436758c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:50.457086 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.439105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.8.57.1:43129)
0929 02:56:50.439262 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.439267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.439285 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:56:50.439354 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.439360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.439445 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.439542 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:50.441180 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.442274 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.444206 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.444809 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.444907 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.450004 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:50.450019 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:50.450037 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:50.450091 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:50.453799 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.454578 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.454587 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.454590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.454747 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.455203 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.455208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.455211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.455610 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:50.455913 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.456471 (+ 558us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.456690 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:56:50.457703 25736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:50.574987 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25613
W20260929 02:56:50.589242 25538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.57.1:43129 (error 108)
I20260929 02:56:50.589408 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25747
I20260929 02:56:50.591130 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.590655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53958)
0929 02:56:50.590851 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.590964 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 02:56:50.591392 25538 catalog_manager.cc:4584] TS aa748292d8664b1fab9118a436758c8f (127.8.57.1:43129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c91ee7185ac74b198e3f12fb7bcbb1d2: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Sep 29 02:56:50 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:56:50.605106 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.590248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.8.57.2:38089)
0929 02:56:50.590416 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.590423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.590507 (+ 84us) server_negotiation.cc:407] Connection header received
0929 02:56:50.590668 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.590674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.590753 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.590860 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:50.592442 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.593264 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.594534 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.594975 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.595084 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.599565 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:50.599576 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:50.599590 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:50.599626 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:50.602452 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.603220 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.603227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.603229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.603352 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.603710 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.603713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.603715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.604152 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:50.604466 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.604666 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.604830 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:56:50.605716 25870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:50.645006 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.644505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53970)
0929 02:56:50.644752 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.644850 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 02:56:50.646019 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.645681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53974)
0929 02:56:50.645823 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.645904 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 02:56:50.706548 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.706051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53986)
0929 02:56:50.706281 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.706377 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260929 02:56:50.707559 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.707207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53988)
0929 02:56:50.707368 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.707443 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 02:56:50.715649 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25747
I20260929 02:56:50.731242 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25880
Sep 29 02:56:50 dist-test-slave-1dkt krb5kdc[25507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:56:50.745877 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.731939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.8.57.3:42493)
0929 02:56:50.732104 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:50.732109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:50.732131 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:56:50.732348 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:50.732354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:50.732433 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:50.732515 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:50.733733 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.734443 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.735691 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:50.736114 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:50.736218 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:50.740396 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:50.740410 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:50.740425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:50.740458 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:50.743109 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.743753 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.743760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.743762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.743937 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:50.744411 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:50.744416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:50.744419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:50.744793 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:50.745072 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:50.745367 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 02:56:50.745563 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:56:50.746467 26003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:50.788447 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.787930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:53998)
0929 02:56:50.788161 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.788258 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 02:56:50.789454 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:50.789086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43129 (local address 127.0.0.1:54004)
0929 02:56:50.789276 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:50.789347 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.1:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 02:56:50.832352 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25880
I20260929 02:56:50.845321 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 25520
I20260929 02:56:50.846454 25582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:50.953504 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 25520
2026-09-29T02:56:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4129 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T02:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:56:51Z Disabled control of system clock
I20260929 02:56:51.020996 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41013
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:56:51.335368 26062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:51.335655 26062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:51.335783 26062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:51.343050 26062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:56:51.343158 26062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:51.343223 26062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:51.343302 26062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:56:51.343362 26062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:56:51.352896 26062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41013
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:51.354092 26062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:51.355726 26062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:51.366761 26067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:51.366797 26068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:51.366751 26070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:51.367194 26062 server_base.cc:1034] running on GCE node
I20260929 02:56:51.368117 26062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:51.369424 26062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:51.370621 26062 hybrid_clock.cc:648] HybridClock initialized: now 1790650611370572 us; error 38 us; skew 500 ppm
I20260929 02:56:51.370987 26062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:51.373067 26062 webserver.cc:460] Webserver started at http://127.8.57.62:46357/ using document root <none> and password file <none>
I20260929 02:56:51.373687 26062 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:51.373787 26062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:51.374075 26062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:51.376850 26062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "18b2f561bd9246c2aa7d7b1232472393"
format_stamp: "Formatted at 2026-09-29 02:56:51 on dist-test-slave-1dkt"
server_key: "6db3c9907e558ac6f85caddc259b32de"
server_key_iv: "9d2cb5e332d1606af6ca9c4919f52d4f"
server_key_version: "encryptionkey@0"
I20260929 02:56:51.377591 26062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "18b2f561bd9246c2aa7d7b1232472393"
format_stamp: "Formatted at 2026-09-29 02:56:51 on dist-test-slave-1dkt"
server_key: "6db3c9907e558ac6f85caddc259b32de"
server_key_iv: "9d2cb5e332d1606af6ca9c4919f52d4f"
server_key_version: "encryptionkey@0"
I20260929 02:56:51.382315 26062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:56:51.385527 26076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:51.386827 26062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:56:51.386981 26062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "18b2f561bd9246c2aa7d7b1232472393"
format_stamp: "Formatted at 2026-09-29 02:56:51 on dist-test-slave-1dkt"
server_key: "6db3c9907e558ac6f85caddc259b32de"
server_key_iv: "9d2cb5e332d1606af6ca9c4919f52d4f"
server_key_version: "encryptionkey@0"
I20260929 02:56:51.387163 26062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:51.406430 26062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:51.407205 26062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:51.407465 26062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:51.423550 26062 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41013
I20260929 02:56:51.423573 26127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41013 every 8 connection(s)
I20260929 02:56:51.424783 26062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:56:51.428208 26128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:51.429633 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26062
I20260929 02:56:51.429819 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:56:51.430263 8420 external_mini_cluster.cc:1428] Setting key 4799e3ba547fa0ecd27687f60fb118f4
I20260929 02:56:51.441751 26128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393: Bootstrap starting.
I20260929 02:56:51.444448 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:51.432010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.8.57.62:41013)
0929 02:56:51.433048 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:51.433062 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:51.433108 (+ 46us) server_negotiation.cc:407] Connection header received
0929 02:56:51.434695 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:51.434719 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:51.435308 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:51.435710 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:51.436209 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:51.438005 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:51.439437 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:51.440444 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:51.440608 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:51.441867 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:51.441878 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:51.441895 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:51.442536 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:51.442628 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:51.442658 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:51.442761 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:51.442948 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:56:51.443248 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":325,"threads_started":1}
I20260929 02:56:51.445397 26128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:51.446413 26128 log.cc:824] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:51.448928 26128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393: No bootstrap required, opened a new log
I20260929 02:56:51.455711 26128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b2f561bd9246c2aa7d7b1232472393" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41013 } }
I20260929 02:56:51.456095 26128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:51.456192 26128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b2f561bd9246c2aa7d7b1232472393, State: Initialized, Role: FOLLOWER
I20260929 02:56:51.456707 26128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b2f561bd9246c2aa7d7b1232472393" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41013 } }
I20260929 02:56:51.456919 26128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:56:51.457039 26128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:56:51.457178 26128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:51.459170 26128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b2f561bd9246c2aa7d7b1232472393" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41013 } }
I20260929 02:56:51.459616 26128 leader_election.cc:302] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18b2f561bd9246c2aa7d7b1232472393; no voters:
I20260929 02:56:51.460160 26128 leader_election.cc:288] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:56:51.460286 26133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:51.461308 26133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [term 1 LEADER]: Becoming Leader. State: Replica: 18b2f561bd9246c2aa7d7b1232472393, State: Running, Role: LEADER
I20260929 02:56:51.461859 26133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b2f561bd9246c2aa7d7b1232472393" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41013 } }
I20260929 02:56:51.462081 26128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:56:51.464591 26135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18b2f561bd9246c2aa7d7b1232472393. Latest consensus state: current_term: 1 leader_uuid: "18b2f561bd9246c2aa7d7b1232472393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b2f561bd9246c2aa7d7b1232472393" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41013 } } }
I20260929 02:56:51.464569 26134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18b2f561bd9246c2aa7d7b1232472393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18b2f561bd9246c2aa7d7b1232472393" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41013 } } }
I20260929 02:56:51.465000 26135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:51.465000 26134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393 [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:51.466454 26142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:56:51.471213 26142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:56:51.479574 26142 catalog_manager.cc:1347] Generated new cluster ID: 764d3e784f6243a58e7b893c4a9d7f0e
I20260929 02:56:51.479682 26142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:56:51.495307 26142 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:56:51.496598 26142 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:56:51.503480 26142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393: Generated new TSK 0
I20260929 02:56:51.504316 26142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:56:51.511466 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41013
--builtin_ntp_servers=127.8.57.20:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:56:51.836897 26152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:51.837153 26152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:51.837276 26152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:51.844262 26152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:51.844357 26152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:51.844506 26152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:56:51.853462 26152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:51.854687 26152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:51.856436 26152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:51.868772 26160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:51.868840 26158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:51.869194 26157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:51.869419 26152 server_base.cc:1034] running on GCE node
I20260929 02:56:51.869982 26152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:51.870818 26152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:51.871970 26152 hybrid_clock.cc:648] HybridClock initialized: now 1790650611871920 us; error 40 us; skew 500 ppm
I20260929 02:56:51.872267 26152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:51.880561 26152 webserver.cc:460] Webserver started at http://127.8.57.1:35993/ using document root <none> and password file <none>
I20260929 02:56:51.881193 26152 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:51.881287 26152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:51.881578 26152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:51.884325 26152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "a11b8f97942c4dae9fc9e1b1175511f4"
format_stamp: "Formatted at 2026-09-29 02:56:51 on dist-test-slave-1dkt"
server_key: "f5fba0697e9e3800e9992ec85af74323"
server_key_iv: "acb19d22a277e849f8acdc07c6b231b6"
server_key_version: "encryptionkey@0"
I20260929 02:56:51.885036 26152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "a11b8f97942c4dae9fc9e1b1175511f4"
format_stamp: "Formatted at 2026-09-29 02:56:51 on dist-test-slave-1dkt"
server_key: "f5fba0697e9e3800e9992ec85af74323"
server_key_iv: "acb19d22a277e849f8acdc07c6b231b6"
server_key_version: "encryptionkey@0"
I20260929 02:56:51.889669 26152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:56:51.893026 26166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:51.894235 26152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:56:51.894379 26152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "a11b8f97942c4dae9fc9e1b1175511f4"
format_stamp: "Formatted at 2026-09-29 02:56:51 on dist-test-slave-1dkt"
server_key: "f5fba0697e9e3800e9992ec85af74323"
server_key_iv: "acb19d22a277e849f8acdc07c6b231b6"
server_key_version: "encryptionkey@0"
I20260929 02:56:51.894560 26152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:51.919643 26152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:51.920476 26152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:51.920727 26152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:51.921622 26152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:51.922977 26152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:51.923065 26152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:51.923161 26152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:51.923231 26152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:51.951229 26152 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:35795
I20260929 02:56:51.951290 26278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:35795 every 8 connection(s)
I20260929 02:56:51.952488 26152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:56:51.956760 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26152
I20260929 02:56:51.956897 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:56:51.957326 8420 external_mini_cluster.cc:1428] Setting key dfd18a4354b4122ac3b304e270dd6909
I20260929 02:56:51.959672 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41013
--builtin_ntp_servers=127.8.57.20:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:56:51.964141 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:51.954736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:48819 (local address 127.8.57.62:41013)
0929 02:56:51.955125 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:51.955129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:51.956293 (+ 1164us) server_negotiation.cc:407] Connection header received
0929 02:56:51.958076 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:51.958082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:51.958157 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:51.958246 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:51.959232 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:51.960011 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:51.961463 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:51.961922 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:51.962033 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:51.963351 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:51.963356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:51.963359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:51.963376 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:51.963413 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:51.963424 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:51.963521 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:51.963799 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 02:56:51.963900 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":202,"threads_started":1}
I20260929 02:56:51.965355 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:51.955260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41013 (local address 127.8.57.1:48819)
0929 02:56:51.956143 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:51.956176 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:51.957833 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:51.958472 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:51.958480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:51.958567 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:51.959079 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:51.959095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:51.960140 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:51.960144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:51.961330 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:51.961342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:51.962154 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:51.962161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:51.962371 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:51.963018 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:56:51.963021 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:51.963078 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:56:51.963170 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:51.963550 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:51.963562 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:51.963571 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:56:51.963751 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 02:56:51.964165 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":258,"threads_started":1}
I20260929 02:56:51.966866 26279 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41013
I20260929 02:56:51.967136 26279 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:51.967904 26279 heartbeater.cc:511] Master 127.8.57.62:41013 requested a full tablet report, sending...
I20260929 02:56:51.969797 26093 ts_manager.cc:194] Registered new tserver with Master: a11b8f97942c4dae9fc9e1b1175511f4 (127.8.57.1:35795)
I20260929 02:56:51.971204 26093 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.1:48819
W20260929 02:56:52.266414 26283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:52.266670 26283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:52.266801 26283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:52.273880 26283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:52.273973 26283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:52.274121 26283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:56:52.283352 26283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:52.284610 26283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:52.286274 26283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:52.298801 26289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:52.298833 26291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:52.298804 26288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:52.299631 26283 server_base.cc:1034] running on GCE node
I20260929 02:56:52.300251 26283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:52.301131 26283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:52.302338 26283 hybrid_clock.cc:648] HybridClock initialized: now 1790650612302296 us; error 34 us; skew 500 ppm
I20260929 02:56:52.302629 26283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:52.304716 26283 webserver.cc:460] Webserver started at http://127.8.57.2:38965/ using document root <none> and password file <none>
I20260929 02:56:52.305367 26283 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:52.305465 26283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:52.305764 26283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:52.308589 26283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "37277f877b9248af9abe53aea7a8aa93"
format_stamp: "Formatted at 2026-09-29 02:56:52 on dist-test-slave-1dkt"
server_key: "93f4ba533d32ec0976f797dc2a96d82b"
server_key_iv: "9737f4b421aa942176d9a09e80b89483"
server_key_version: "encryptionkey@0"
I20260929 02:56:52.309326 26283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "37277f877b9248af9abe53aea7a8aa93"
format_stamp: "Formatted at 2026-09-29 02:56:52 on dist-test-slave-1dkt"
server_key: "93f4ba533d32ec0976f797dc2a96d82b"
server_key_iv: "9737f4b421aa942176d9a09e80b89483"
server_key_version: "encryptionkey@0"
I20260929 02:56:52.314077 26283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:56:52.317237 26297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:52.318442 26283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:56:52.318591 26283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "37277f877b9248af9abe53aea7a8aa93"
format_stamp: "Formatted at 2026-09-29 02:56:52 on dist-test-slave-1dkt"
server_key: "93f4ba533d32ec0976f797dc2a96d82b"
server_key_iv: "9737f4b421aa942176d9a09e80b89483"
server_key_version: "encryptionkey@0"
I20260929 02:56:52.318782 26283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:52.332906 26283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:52.333599 26283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:52.333858 26283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:52.334707 26283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:52.336092 26283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:52.336184 26283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:52.336297 26283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:52.336364 26283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:52.364405 26283 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:39281
I20260929 02:56:52.364447 26409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:39281 every 8 connection(s)
I20260929 02:56:52.365610 26283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:56:52.376205 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26283
I20260929 02:56:52.376391 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:56:52.376906 8420 external_mini_cluster.cc:1428] Setting key b9de90791718c6235cddbdf600bcf201
I20260929 02:56:52.377883 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.367984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:34337 (local address 127.8.57.62:41013)
0929 02:56:52.368181 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.368186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.369428 (+ 1242us) server_negotiation.cc:407] Connection header received
0929 02:56:52.371064 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.371069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.371133 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.371215 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:52.372299 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.373033 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.375212 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.375608 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.375740 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.377095 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:52.377100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:52.377103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:52.377120 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:52.377154 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:52.377166 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:52.377234 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.377485 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.377581 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:56:52.378801 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.368468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41013 (local address 127.8.57.2:34337)
0929 02:56:52.369291 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.369324 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.370819 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.371376 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.371384 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.371460 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:52.372135 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.372160 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.373167 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.373173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.375073 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.375086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.375804 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.375811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.376122 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.376781 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:56:52.376784 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:52.376840 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:56:52.376928 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:52.377253 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:52.377263 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:52.377270 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.377435 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.377731 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":252,"threads_started":1}
I20260929 02:56:52.379722 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41013
--builtin_ntp_servers=127.8.57.20:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:56:52.380115 26410 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41013
I20260929 02:56:52.380390 26410 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:52.381204 26410 heartbeater.cc:511] Master 127.8.57.62:41013 requested a full tablet report, sending...
I20260929 02:56:52.382519 26093 ts_manager.cc:194] Registered new tserver with Master: 37277f877b9248af9abe53aea7a8aa93 (127.8.57.2:39281)
I20260929 02:56:52.383282 26093 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.2:34337
W20260929 02:56:52.687968 26413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:52.688220 26413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:52.688351 26413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:52.695410 26413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:52.695497 26413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:52.695626 26413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:56:52.704468 26413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:52.705654 26413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:52.707304 26413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:52.719401 26421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:52.719280 26419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:52.719297 26418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:52.719944 26413 server_base.cc:1034] running on GCE node
I20260929 02:56:52.720530 26413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:52.721431 26413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:52.722621 26413 hybrid_clock.cc:648] HybridClock initialized: now 1790650612722572 us; error 36 us; skew 500 ppm
I20260929 02:56:52.722926 26413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:52.724989 26413 webserver.cc:460] Webserver started at http://127.8.57.3:43403/ using document root <none> and password file <none>
I20260929 02:56:52.725595 26413 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:52.725690 26413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:52.725979 26413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:52.728642 26413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "77e1b5b503b545989315f77ab169a9fd"
format_stamp: "Formatted at 2026-09-29 02:56:52 on dist-test-slave-1dkt"
server_key: "1101c661c4e75ab7e3642903987b025d"
server_key_iv: "ace2f839399db7532d3611a2698591b5"
server_key_version: "encryptionkey@0"
I20260929 02:56:52.729363 26413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "77e1b5b503b545989315f77ab169a9fd"
format_stamp: "Formatted at 2026-09-29 02:56:52 on dist-test-slave-1dkt"
server_key: "1101c661c4e75ab7e3642903987b025d"
server_key_iv: "ace2f839399db7532d3611a2698591b5"
server_key_version: "encryptionkey@0"
I20260929 02:56:52.733933 26413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:56:52.736986 26427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:52.738148 26413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:56:52.738286 26413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "77e1b5b503b545989315f77ab169a9fd"
format_stamp: "Formatted at 2026-09-29 02:56:52 on dist-test-slave-1dkt"
server_key: "1101c661c4e75ab7e3642903987b025d"
server_key_iv: "ace2f839399db7532d3611a2698591b5"
server_key_version: "encryptionkey@0"
I20260929 02:56:52.738472 26413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:52.759042 26413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:52.759821 26413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:52.760102 26413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:52.761054 26413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:52.762519 26413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:52.762609 26413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:52.762712 26413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:52.762784 26413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:52.791853 26413 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:45671
I20260929 02:56:52.791962 26539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:45671 every 8 connection(s)
I20260929 02:56:52.793083 26413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:56:52.801806 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26413
I20260929 02:56:52.801977 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:56:52.802515 8420 external_mini_cluster.cc:1428] Setting key 3b2bec4beecd709dc94e0329b2512877
I20260929 02:56:52.805102 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.795474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:60347 (local address 127.8.57.62:41013)
0929 02:56:52.795688 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.795694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.797053 (+ 1359us) server_negotiation.cc:407] Connection header received
0929 02:56:52.798851 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.798855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.798922 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.798995 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:52.799952 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.800653 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.802467 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.802904 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.802995 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.804377 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:52.804383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:52.804386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:52.804402 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:52.804432 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:52.804444 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:52.804509 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.804732 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.804821 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:56:52.806030 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.796056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41013 (local address 127.8.57.3:60347)
0929 02:56:52.796913 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.796946 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.798608 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.799162 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.799170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.799252 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:52.799768 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.799783 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.800792 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.800796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.802347 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.802361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.803112 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.803118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.803344 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.804095 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:56:52.804099 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:52.804142 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:56:52.804217 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:52.804530 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:52.804540 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:52.804546 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.804711 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.805104 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":261,"threads_started":1}
I20260929 02:56:52.807128 26540 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41013
I20260929 02:56:52.807364 26540 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:52.808045 26540 heartbeater.cc:511] Master 127.8.57.62:41013 requested a full tablet report, sending...
I20260929 02:56:52.809257 26093 ts_manager.cc:194] Registered new tserver with Master: 77e1b5b503b545989315f77ab169a9fd (127.8.57.3:45671)
I20260929 02:56:52.809983 26093 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.3:60347
I20260929 02:56:52.817245 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:56:52.826331 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.821069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.8.57.62:41013)
0929 02:56:52.821212 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.821216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.821626 (+ 410us) server_negotiation.cc:407] Connection header received
0929 02:56:52.821776 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.821780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.821837 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.821908 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:52.822325 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.823018 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.824470 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.824857 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.824943 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.825813 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:52.825817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:52.825820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:52.825836 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:52.825865 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:52.825876 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:52.825939 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.826035 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.826111 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:56:52.829777 26093 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:56:52.836287 26093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:56:52.856801 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.847562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:35795 (local address 127.0.0.1:52954)
0929 02:56:52.849126 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.849170 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.849351 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.850280 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.850286 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.850315 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:52.850679 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.850696 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.852346 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.852352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.854336 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.854348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.855922 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.855928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.856122 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.856167 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.856282 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.856458 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1328,"mutex_wait_us":3,"thread_start_us":1195,"threads_started":1}
I20260929 02:56:52.856845 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.848035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.8.57.1:35795)
0929 02:56:52.848687 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.848705 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.849288 (+ 583us) server_negotiation.cc:407] Connection header received
0929 02:56:52.849577 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.849586 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.849832 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.850010 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:52.850905 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.852232 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.854556 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.855708 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.855847 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.856125 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.856355 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.856523 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":410,"threads_started":1}
I20260929 02:56:52.857738 26553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.847598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45671 (local address 127.0.0.1:60608)
0929 02:56:52.849884 (+ 2286us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.849901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.850012 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.850874 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.850878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.850899 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:52.851243 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.851255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.852978 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.852983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.855109 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.855123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.857085 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.857092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.857208 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.857230 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.857305 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.857427 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2157,"spinlock_wait_cycles":15232,"thread_start_us":2833,"threads_started":1}
I20260929 02:56:52.857801 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.847840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60608 (local address 127.8.57.3:45671)
0929 02:56:52.849165 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.849173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.850148 (+ 975us) server_negotiation.cc:407] Connection header received
0929 02:56:52.850242 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.850249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.850468 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.850632 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:52.851447 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.852866 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.855297 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.856615 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.856732 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.856974 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.857360 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.857505 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":998,"threads_started":1}
I20260929 02:56:52.858709 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.849855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:39281 (local address 127.0.0.1:54744)
0929 02:56:52.851314 (+ 1459us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.851329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.851416 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.852402 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.852406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.852431 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:52.852821 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.852833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.854700 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.854705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.856384 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.856397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.858064 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.858070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.858236 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.858270 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.858342 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.858448 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":741,"threads_started":1}
I20260929 02:56:52.859766 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.850242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54744 (local address 127.8.57.2:39281)
0929 02:56:52.851859 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.851867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.851921 (+ 54us) server_negotiation.cc:407] Connection header received
0929 02:56:52.852017 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.852024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.852200 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.852415 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:52.853030 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.854543 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.856564 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.857880 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.858968 (+ 1088us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.859210 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.859290 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.859432 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1450,"thread_start_us":1327,"threads_started":1}
I20260929 02:56:52.861156 26475 tablet_service.cc:1467] Processing CreateTablet for tablet 1bb7287a4f0a4331bfae4a2db4b9de64 (DEFAULT_TABLE table=test-table [id=a62c60ffd3e7440cabf92b8559e34ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:52.861129 26214 tablet_service.cc:1467] Processing CreateTablet for tablet 1bb7287a4f0a4331bfae4a2db4b9de64 (DEFAULT_TABLE table=test-table [id=a62c60ffd3e7440cabf92b8559e34ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:52.862262 26475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bb7287a4f0a4331bfae4a2db4b9de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:52.862421 26214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bb7287a4f0a4331bfae4a2db4b9de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:52.863081 26345 tablet_service.cc:1467] Processing CreateTablet for tablet 1bb7287a4f0a4331bfae4a2db4b9de64 (DEFAULT_TABLE table=test-table [id=a62c60ffd3e7440cabf92b8559e34ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:52.864300 26345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bb7287a4f0a4331bfae4a2db4b9de64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:52.872936 26558 tablet_bootstrap.cc:492] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Bootstrap starting.
I20260929 02:56:52.875888 26559 tablet_bootstrap.cc:492] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Bootstrap starting.
I20260929 02:56:52.877249 26558 tablet_bootstrap.cc:654] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:52.877322 26560 tablet_bootstrap.cc:492] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Bootstrap starting.
I20260929 02:56:52.878810 26558 log.cc:824] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:52.879138 26559 tablet_bootstrap.cc:654] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:52.880148 26560 tablet_bootstrap.cc:654] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:52.880239 26559 log.cc:824] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:52.881211 26560 log.cc:824] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:52.881937 26558 tablet_bootstrap.cc:492] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: No bootstrap required, opened a new log
I20260929 02:56:52.882159 26558 ts_tablet_manager.cc:1397] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260929 02:56:52.882630 26559 tablet_bootstrap.cc:492] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: No bootstrap required, opened a new log
I20260929 02:56:52.882848 26559 ts_tablet_manager.cc:1397] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:56:52.883792 26560 tablet_bootstrap.cc:492] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: No bootstrap required, opened a new log
I20260929 02:56:52.884063 26560 ts_tablet_manager.cc:1397] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 02:56:52.889762 26559 raft_consensus.cc:348] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.890151 26559 raft_consensus.cc:374] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:52.890239 26559 raft_consensus.cc:729] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a11b8f97942c4dae9fc9e1b1175511f4, State: Initialized, Role: FOLLOWER
I20260929 02:56:52.890772 26559 consensus_queue.cc:260] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.891812 26279 heartbeater.cc:503] Master 127.8.57.62:41013 was elected leader, sending a full tablet report...
I20260929 02:56:52.892441 26559 ts_tablet_manager.cc:1428] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Time spent starting tablet: real 0.009s user 0.011s sys 0.000s
I20260929 02:56:52.892190 26558 raft_consensus.cc:348] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.892586 26558 raft_consensus.cc:374] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:52.892663 26558 raft_consensus.cc:729] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e1b5b503b545989315f77ab169a9fd, State: Initialized, Role: FOLLOWER
I20260929 02:56:52.893193 26558 consensus_queue.cc:260] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.893895 26560 raft_consensus.cc:348] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.894286 26560 raft_consensus.cc:374] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:52.894395 26560 raft_consensus.cc:729] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37277f877b9248af9abe53aea7a8aa93, State: Initialized, Role: FOLLOWER
I20260929 02:56:52.894636 26540 heartbeater.cc:503] Master 127.8.57.62:41013 was elected leader, sending a full tablet report...
I20260929 02:56:52.894760 26558 ts_tablet_manager.cc:1428] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Time spent starting tablet: real 0.012s user 0.000s sys 0.011s
I20260929 02:56:52.895010 26560 consensus_queue.cc:260] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.895992 26410 heartbeater.cc:503] Master 127.8.57.62:41013 was elected leader, sending a full tablet report...
I20260929 02:56:52.896571 26560 ts_tablet_manager.cc:1428] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260929 02:56:52.911078 26566 raft_consensus.cc:482] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:56:52.911304 26566 raft_consensus.cc:504] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.912721 26566 leader_election.cc:288] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a11b8f97942c4dae9fc9e1b1175511f4 (127.8.57.1:35795), 77e1b5b503b545989315f77ab169a9fd (127.8.57.3:45671)
I20260929 02:56:52.920967 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.913182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:41075 (local address 127.8.57.1:35795)
0929 02:56:52.913365 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.913372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.913922 (+ 550us) server_negotiation.cc:407] Connection header received
0929 02:56:52.914116 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.914121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.914205 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.914295 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:52.914977 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.916136 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.918898 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.920080 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.920203 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.920292 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.920599 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.920718 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:56:52.920995 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.913182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:50345 (local address 127.8.57.3:45671)
0929 02:56:52.913376 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.913383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.913918 (+ 535us) server_negotiation.cc:407] Connection header received
0929 02:56:52.914123 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.914128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.914198 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.914252 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:52.915064 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.916127 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.918964 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.920203 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.920305 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.920387 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.920655 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.920763 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:56:52.921056 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.912950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:35795 (local address 127.8.57.2:41075)
0929 02:56:52.913804 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.913823 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.913966 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.914333 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.914337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.914364 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:52.914815 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.914828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.916250 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.916255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.918756 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.918771 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.920208 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.920213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.920417 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.920455 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.920545 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.920666 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":179,"spinlock_wait_cycles":18688,"thread_start_us":1095,"threads_started":1}
I20260929 02:56:52.921379 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.912950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:45671 (local address 127.8.57.2:50345)
0929 02:56:52.913842 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:52.913856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:52.913989 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:52.914417 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:52.914421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:52.914439 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:52.914913 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.914924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.916252 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.916256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.918761 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:52.918771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.920366 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.920371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:52.920529 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.920557 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 02:56:52.920626 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 02:56:52.920714 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":181,"thread_start_us":392,"threads_started":1}
I20260929 02:56:52.921869 26234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb7287a4f0a4331bfae4a2db4b9de64" candidate_uuid: "37277f877b9248af9abe53aea7a8aa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" is_pre_election: true
I20260929 02:56:52.922297 26234 raft_consensus.cc:2454] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37277f877b9248af9abe53aea7a8aa93 in term 0.
I20260929 02:56:52.922159 26495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb7287a4f0a4331bfae4a2db4b9de64" candidate_uuid: "37277f877b9248af9abe53aea7a8aa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e1b5b503b545989315f77ab169a9fd" is_pre_election: true
I20260929 02:56:52.922652 26495 raft_consensus.cc:2454] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37277f877b9248af9abe53aea7a8aa93 in term 0.
I20260929 02:56:52.922879 26299 leader_election.cc:302] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37277f877b9248af9abe53aea7a8aa93, a11b8f97942c4dae9fc9e1b1175511f4; no voters:
I20260929 02:56:52.923206 26566 raft_consensus.cc:2781] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:56:52.923357 26566 raft_consensus.cc:482] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:56:52.923449 26566 raft_consensus.cc:3037] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:52.925690 26566 raft_consensus.cc:504] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.926195 26566 leader_election.cc:288] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [CANDIDATE]: Term 1 election: Requested vote from peers a11b8f97942c4dae9fc9e1b1175511f4 (127.8.57.1:35795), 77e1b5b503b545989315f77ab169a9fd (127.8.57.3:45671)
I20260929 02:56:52.926589 26234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb7287a4f0a4331bfae4a2db4b9de64" candidate_uuid: "37277f877b9248af9abe53aea7a8aa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a11b8f97942c4dae9fc9e1b1175511f4"
I20260929 02:56:52.926664 26495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb7287a4f0a4331bfae4a2db4b9de64" candidate_uuid: "37277f877b9248af9abe53aea7a8aa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e1b5b503b545989315f77ab169a9fd"
I20260929 02:56:52.926750 26234 raft_consensus.cc:3037] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:52.926841 26495 raft_consensus.cc:3037] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:52.928941 26234 raft_consensus.cc:2454] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37277f877b9248af9abe53aea7a8aa93 in term 1.
I20260929 02:56:52.929008 26495 raft_consensus.cc:2454] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37277f877b9248af9abe53aea7a8aa93 in term 1.
I20260929 02:56:52.929256 26299 leader_election.cc:302] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37277f877b9248af9abe53aea7a8aa93, a11b8f97942c4dae9fc9e1b1175511f4; no voters:
I20260929 02:56:52.929497 26566 raft_consensus.cc:2781] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:52.929912 26566 raft_consensus.cc:686] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 1 LEADER]: Becoming Leader. State: Replica: 37277f877b9248af9abe53aea7a8aa93, State: Running, Role: LEADER
I20260929 02:56:52.930334 26566 consensus_queue.cc:237] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } }
I20260929 02:56:52.934494 26092 catalog_manager.cc:5495] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37277f877b9248af9abe53aea7a8aa93 (127.8.57.2). New cstate: current_term: 1 leader_uuid: "37277f877b9248af9abe53aea7a8aa93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37277f877b9248af9abe53aea7a8aa93" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 39281 } health_report { overall_health: HEALTHY } } }
W20260929 02:56:52.954455 26280 tablet.cc:2369] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:52.955204 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.949570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54756 (local address 127.8.57.2:39281)
0929 02:56:52.949743 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.949749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:52.949766 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:56:52.949955 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:52.949959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:52.950032 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:52.950146 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:56:52.950506 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.951493 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.952887 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:52.953495 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:52.953598 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:52.954046 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:56:52.954617 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:56:52.954746 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:52.954884 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 02:56:52.954965 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:56:52.962904 26495 raft_consensus.cc:1261] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 1 FOLLOWER]: Refusing update from remote peer 37277f877b9248af9abe53aea7a8aa93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:52.962942 26234 raft_consensus.cc:1261] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 1 FOLLOWER]: Refusing update from remote peer 37277f877b9248af9abe53aea7a8aa93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:52.963780 26569 consensus_queue.cc:1035] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a11b8f97942c4dae9fc9e1b1175511f4" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 35795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:52.964212 26566 consensus_queue.cc:1035] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e1b5b503b545989315f77ab169a9fd" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 45671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:52.973526 26573 mvcc.cc:205] Tried to move back new op lower bound from 7334504910686851072 to 7334504910566051840. Current Snapshot: MvccSnapshot[applied={T|T < 7334504910686851072}]
I20260929 02:56:52.987957 26093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:52.988240 26093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:52.991755 26093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18b2f561bd9246c2aa7d7b1232472393: Sending DeleteTablet for 3 replicas of tablet 1bb7287a4f0a4331bfae4a2db4b9de64
I20260929 02:56:52.992823 26214 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bb7287a4f0a4331bfae4a2db4b9de64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:52 UTC) from {username='slave'} at 127.0.0.1:52954
I20260929 02:56:52.992905 26475 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bb7287a4f0a4331bfae4a2db4b9de64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:52 UTC) from {username='slave'} at 127.0.0.1:60608
I20260929 02:56:52.993693 26583 tablet_replica.cc:330] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: stopping tablet replica
I20260929 02:56:52.993884 26345 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bb7287a4f0a4331bfae4a2db4b9de64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:52 UTC) from {username='slave'} at 127.0.0.1:54744
I20260929 02:56:52.994212 26583 raft_consensus.cc:2229] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:52.994642 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26152
I20260929 02:56:52.994755 26583 raft_consensus.cc:2258] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:52.995522 26582 tablet_replica.cc:330] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: stopping tablet replica
I20260929 02:56:52.996021 26582 raft_consensus.cc:2229] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:52.996534 26582 raft_consensus.cc:2258] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:52.996937 26583 ts_tablet_manager.cc:1905] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:53.000322 26584 tablet_replica.cc:330] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: stopping tablet replica
I20260929 02:56:53.000672 26584 raft_consensus.cc:2229] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:56:53.001241 26584 raft_consensus.cc:2258] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:53.002394 26583 ts_tablet_manager.cc:1918] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:53.002542 26583 log.cc:1196] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/1bb7287a4f0a4331bfae4a2db4b9de64
I20260929 02:56:53.003023 26583 ts_tablet_manager.cc:1939] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 77e1b5b503b545989315f77ab169a9fd: Deleting consensus metadata
I20260929 02:56:53.003144 26584 ts_tablet_manager.cc:1905] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:53.004324 26582 ts_tablet_manager.cc:1905] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:53.004688 26077 catalog_manager.cc:4841] TS 77e1b5b503b545989315f77ab169a9fd (127.8.57.3:45671): tablet 1bb7287a4f0a4331bfae4a2db4b9de64 (table test-table [id=a62c60ffd3e7440cabf92b8559e34ceb]) successfully deleted
I20260929 02:56:53.010094 26584 ts_tablet_manager.cc:1918] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:53.010265 26584 log.cc:1196] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/1bb7287a4f0a4331bfae4a2db4b9de64
I20260929 02:56:53.009663 26582 ts_tablet_manager.cc:1918] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:53.010535 26582 log.cc:1196] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/1bb7287a4f0a4331bfae4a2db4b9de64
I20260929 02:56:53.010766 26584 ts_tablet_manager.cc:1939] T 1bb7287a4f0a4331bfae4a2db4b9de64 P 37277f877b9248af9abe53aea7a8aa93: Deleting consensus metadata
I20260929 02:56:53.011133 26582 ts_tablet_manager.cc:1939] T 1bb7287a4f0a4331bfae4a2db4b9de64 P a11b8f97942c4dae9fc9e1b1175511f4: Deleting consensus metadata
I20260929 02:56:53.012449 26078 catalog_manager.cc:4841] TS 37277f877b9248af9abe53aea7a8aa93 (127.8.57.2:39281): tablet 1bb7287a4f0a4331bfae4a2db4b9de64 (table test-table [id=a62c60ffd3e7440cabf92b8559e34ceb]) successfully deleted
I20260929 02:56:53.012709 26078 catalog_manager.cc:4841] TS a11b8f97942c4dae9fc9e1b1175511f4 (127.8.57.1:35795): tablet 1bb7287a4f0a4331bfae4a2db4b9de64 (table test-table [id=a62c60ffd3e7440cabf92b8559e34ceb]) successfully deleted
I20260929 02:56:53.016510 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:52.996092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.8.57.1:35795)
0929 02:56:52.996285 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:52.996292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:53.008397 (+ 12105us) server_negotiation.cc:407] Connection header received
0929 02:56:53.008671 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:53.008677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:53.008767 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:53.008830 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:53.009472 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:53.010360 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:53.012276 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:53.012861 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:53.012975 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:53.014725 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:53.014736 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:53.014753 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:53.015460 (+ 707us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:53.015520 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:53.015551 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:53.015654 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:53.015964 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 02:56:53.016251 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:56:53.017074 26274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:53.106093 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26152
I20260929 02:56:53.125664 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26283
I20260929 02:56:53.133169 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:53.126400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54764 (local address 127.8.57.2:39281)
0929 02:56:53.126600 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:53.126605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:53.126626 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:56:53.126726 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:53.126731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:53.126844 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:53.126946 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:53.127264 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:53.128440 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:53.129656 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:53.130299 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:53.130432 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:53.131496 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:53.131507 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:53.131524 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:53.132355 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:53.132418 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:53.132442 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:53.132547 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:53.132694 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 02:56:53.132793 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:56:53.135109 26405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:53.245096 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26283
I20260929 02:56:53.258915 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26413
I20260929 02:56:53.265268 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:53.259653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.8.57.3:45671)
0929 02:56:53.259806 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:53.259810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:53.259828 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:56:53.260003 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:53.260007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:53.260087 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:53.260161 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:53.260492 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:53.261230 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:53.262495 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:53.262915 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:53.263000 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:53.264018 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:53.264026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:53.264039 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:53.264582 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:53.264631 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:53.264649 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:53.264728 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:53.264895 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 02:56:53.264975 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:56:53.265935 26535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:53.351258 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26413
I20260929 02:56:53.363773 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26062
I20260929 02:56:53.364868 26123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:53.469688 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26062
2026-09-29T02:56:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2514 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:56:53 dist-test-slave-1dkt krb5kdc[26592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:56:53 dist-test-slave-1dkt krb5kdc[26592](info): set up 2 sockets
Sep 29 02:56:53 dist-test-slave-1dkt krb5kdc[26592](info): commencing operation
krb5kdc: starting...
W20260929 02:56:54.475739 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:56:54 dist-test-slave-1dkt krb5kdc[26592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:56:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:56:54.689332 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41511
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:54.996014 26605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:54.996296 26605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:54.996520 26605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:55.003643 26605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:56:55.003768 26605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:55.003850 26605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:55.003954 26605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:56:55.004011 26605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:56:55.013665 26605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:55.014951 26605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:55.016273 26605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:55.028065 26610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:55.028142 26611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:55.028108 26613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:55.029136 26605 server_base.cc:1034] running on GCE node
I20260929 02:56:55.029860 26605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:55.031121 26605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:55.032308 26605 hybrid_clock.cc:648] HybridClock initialized: now 1790650615032249 us; error 49 us; skew 500 ppm
I20260929 02:56:55.032629 26605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:55 dist-test-slave-1dkt krb5kdc[26592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650615, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:55.035576 26605 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:56:55.036906 26605 webserver.cc:460] Webserver started at http://127.8.57.62:33571/ using document root <none> and password file <none>
I20260929 02:56:55.037509 26605 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:55.037598 26605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:55.037830 26605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:55.040617 26605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "766a9aefaeb74df191b5a4dd95e59d7c"
format_stamp: "Formatted at 2026-09-29 02:56:55 on dist-test-slave-1dkt"
server_key: "1dd7e0f497ba73d5f119440ed5d56c7d"
server_key_iv: "63804eb8cfa0bd0526c40acadd97c442"
server_key_version: "encryptionkey@0"
I20260929 02:56:55.041272 26605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "766a9aefaeb74df191b5a4dd95e59d7c"
format_stamp: "Formatted at 2026-09-29 02:56:55 on dist-test-slave-1dkt"
server_key: "1dd7e0f497ba73d5f119440ed5d56c7d"
server_key_iv: "63804eb8cfa0bd0526c40acadd97c442"
server_key_version: "encryptionkey@0"
I20260929 02:56:55.045967 26605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:56:55.049117 26620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:55.050319 26605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:55.050462 26605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "766a9aefaeb74df191b5a4dd95e59d7c"
format_stamp: "Formatted at 2026-09-29 02:56:55 on dist-test-slave-1dkt"
server_key: "1dd7e0f497ba73d5f119440ed5d56c7d"
server_key_iv: "63804eb8cfa0bd0526c40acadd97c442"
server_key_version: "encryptionkey@0"
I20260929 02:56:55.050622 26605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:55.065490 26605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:55.069325 26605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:55.069572 26605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:55.086002 26605 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41511
I20260929 02:56:55.086055 26671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41511 every 8 connection(s)
I20260929 02:56:55.087242 26605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:56:55.088773 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26605
I20260929 02:56:55.088971 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:56:55.089407 8420 external_mini_cluster.cc:1428] Setting key 37fdcadebd9059ffdb336e24ffff4657
I20260929 02:56:55.091149 26672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:56:55 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:55.103998 26672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c: Bootstrap starting.
I20260929 02:56:55.107053 26672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:55.108268 26672 log.cc:824] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:55.110955 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:55.091033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.8.57.62:41511)
0929 02:56:55.091734 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:55.091745 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:55.091784 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:56:55.093401 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:55.093420 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:55.093774 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:55.094115 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:55.095773 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:55.097116 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:55.098912 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:55.099553 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:55.099671 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:55.104328 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:55.104335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:55.104347 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:55.104386 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:55.107122 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:55.107925 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:55.107932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:55.107937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:55.108099 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:55.108501 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:55.108504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:55.108506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:55.109132 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:55.109416 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:55.109721 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 02:56:55.110015 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":328,"threads_started":1}
I20260929 02:56:55.111212 26672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c: No bootstrap required, opened a new log
I20260929 02:56:55.118851 26672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41511 } }
I20260929 02:56:55.119197 26672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:55.119293 26672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766a9aefaeb74df191b5a4dd95e59d7c, State: Initialized, Role: FOLLOWER
I20260929 02:56:55.119903 26672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41511 } }
I20260929 02:56:55.120126 26672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:56:55.120249 26672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:56:55.120396 26672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:55.122413 26672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41511 } }
I20260929 02:56:55.122972 26672 leader_election.cc:302] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 766a9aefaeb74df191b5a4dd95e59d7c; no voters:
I20260929 02:56:55.123592 26672 leader_election.cc:288] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:56:55.123699 26677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:55.124305 26677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [term 1 LEADER]: Becoming Leader. State: Replica: 766a9aefaeb74df191b5a4dd95e59d7c, State: Running, Role: LEADER
I20260929 02:56:55.124775 26677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41511 } }
I20260929 02:56:55.125434 26672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:56:55.127166 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41511 } } }
I20260929 02:56:55.127521 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:55.129611 26683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:56:55.129482 26679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 766a9aefaeb74df191b5a4dd95e59d7c. Latest consensus state: current_term: 1 leader_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766a9aefaeb74df191b5a4dd95e59d7c" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41511 } } }
I20260929 02:56:55.129757 26679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:55.134179 26683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:56:55.143464 26683 catalog_manager.cc:1347] Generated new cluster ID: 7946da2c92c740148944f976276ed499
I20260929 02:56:55.143563 26683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:56:55.152264 26683 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:56:55.153518 26683 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:56:55.159622 26683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c: Generated new TSK 0
I20260929 02:56:55.160341 26683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:56:55.222801 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41511
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:55.541771 26698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:55.542018 26698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:55.542143 26698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:55.549167 26698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:55.549261 26698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:55.549418 26698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:56:55.558844 26698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:55.560031 26698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:55.561287 26698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:55.573154 26706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:55.573189 26704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:55.573640 26703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:55.574198 26698 server_base.cc:1034] running on GCE node
I20260929 02:56:55.574783 26698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:55.575681 26698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:55.576890 26698 hybrid_clock.cc:648] HybridClock initialized: now 1790650615576844 us; error 67 us; skew 500 ppm
I20260929 02:56:55.577199 26698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:55 dist-test-slave-1dkt krb5kdc[26592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650615, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:55.580149 26698 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:56:55.581547 26698 webserver.cc:460] Webserver started at http://127.8.57.1:40101/ using document root <none> and password file <none>
I20260929 02:56:55.582175 26698 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:55.582274 26698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:55.582563 26698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:55.585390 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "2924d69a699e42eb996f860dd931fa41"
format_stamp: "Formatted at 2026-09-29 02:56:55 on dist-test-slave-1dkt"
server_key: "28df81b2107962022a52c790b139d8ed"
server_key_iv: "d7060f98bd08873baaf543ee3d2e1cfa"
server_key_version: "encryptionkey@0"
I20260929 02:56:55.586122 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "2924d69a699e42eb996f860dd931fa41"
format_stamp: "Formatted at 2026-09-29 02:56:55 on dist-test-slave-1dkt"
server_key: "28df81b2107962022a52c790b139d8ed"
server_key_iv: "d7060f98bd08873baaf543ee3d2e1cfa"
server_key_version: "encryptionkey@0"
I20260929 02:56:55.590932 26698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:56:55.594156 26713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:55.595429 26698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:56:55.595556 26698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "2924d69a699e42eb996f860dd931fa41"
format_stamp: "Formatted at 2026-09-29 02:56:55 on dist-test-slave-1dkt"
server_key: "28df81b2107962022a52c790b139d8ed"
server_key_iv: "d7060f98bd08873baaf543ee3d2e1cfa"
server_key_version: "encryptionkey@0"
I20260929 02:56:55.595739 26698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:55.630717 26698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:55.634954 26698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:55.635214 26698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:55.636195 26698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:55.637506 26698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:55.637593 26698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:55.637712 26698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:55.637769 26698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:55.668077 26698 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:43813
I20260929 02:56:55.668133 26825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:43813 every 8 connection(s)
I20260929 02:56:55.669293 26698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:56:55.677191 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26698
I20260929 02:56:55.677398 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:56:55.677834 8420 external_mini_cluster.cc:1428] Setting key 02f5ab983a5348280078edba9b13f2c7
Sep 29 02:56:55 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650615, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:55.688908 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:55.671571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:55209 (local address 127.8.57.62:41511)
0929 02:56:55.672059 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:55.672063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:55.672878 (+ 815us) server_negotiation.cc:407] Connection header received
0929 02:56:55.674668 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:55.674674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:55.674750 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:55.674849 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:55.676360 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:55.677234 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:55.679066 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:55.679576 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:55.679686 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:55.683152 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:55.683157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:55.683161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:55.683198 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:55.686011 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:55.686649 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:55.686654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:55.686657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:55.686788 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:55.687151 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:55.687155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:55.687158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:55.687473 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:55.687610 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:55.688328 (+ 718us) server_negotiation.cc:299] Negotiation successful
0929 02:56:55.688564 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":389,"thread_start_us":270,"threads_started":1}
I20260929 02:56:55.689792 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:55.672069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41511 (local address 127.8.57.1:55209)
0929 02:56:55.672724 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:55.672759 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:55.674403 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:55.675096 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:55.675104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:55.675596 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:55.676189 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:55.676208 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:55.677410 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:55.677414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:55.678901 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:55.678916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:55.679788 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:55.679794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:55.680052 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:55.680692 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:55.680718 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:55.682941 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:55.686131 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:55.686138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:55.686150 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:55.686538 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:55.686887 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:55.686890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:55.686892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:55.687056 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:55.687570 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:55.687575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:55.687928 (+ 353us) client_negotiation.cc:769] Sending connection context
0929 02:56:55.688185 (+ 257us) client_negotiation.cc:240] Negotiation successful
0929 02:56:55.688676 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":261,"threads_started":1}
I20260929 02:56:55.691237 26826 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41511
I20260929 02:56:55.691538 26826 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:55.692351 26826 heartbeater.cc:511] Master 127.8.57.62:41511 requested a full tablet report, sending...
I20260929 02:56:55.694420 26637 ts_manager.cc:194] Registered new tserver with Master: 2924d69a699e42eb996f860dd931fa41 (127.8.57.1:43813)
I20260929 02:56:55.695906 26637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:55209
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:56:55.727150 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41511
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:56.031949 26832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:56.032214 26832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:56.032348 26832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:56.039724 26832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:56.039848 26832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:56.040030 26832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:56:56.049624 26832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:56.050854 26832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:56.052203 26832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:56.064414 26837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:56.064433 26838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:56.064520 26840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:56.065056 26832 server_base.cc:1034] running on GCE node
I20260929 02:56:56.065660 26832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:56.066540 26832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:56.067745 26832 hybrid_clock.cc:648] HybridClock initialized: now 1790650616067687 us; error 49 us; skew 500 ppm
I20260929 02:56:56.068109 26832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:56 dist-test-slave-1dkt krb5kdc[26592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650616, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:56.070956 26832 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:56:56.072286 26832 webserver.cc:460] Webserver started at http://127.8.57.2:44947/ using document root <none> and password file <none>
I20260929 02:56:56.072916 26832 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:56.073024 26832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:56.073287 26832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:56.076009 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "96aaf34e642e4db9be06df150807f696"
format_stamp: "Formatted at 2026-09-29 02:56:56 on dist-test-slave-1dkt"
server_key: "e7fb4578ac7d09ba38db7acd0fccc7cc"
server_key_iv: "6acc25a7f502eaa6fe6872c19209c72f"
server_key_version: "encryptionkey@0"
I20260929 02:56:56.076717 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "96aaf34e642e4db9be06df150807f696"
format_stamp: "Formatted at 2026-09-29 02:56:56 on dist-test-slave-1dkt"
server_key: "e7fb4578ac7d09ba38db7acd0fccc7cc"
server_key_iv: "6acc25a7f502eaa6fe6872c19209c72f"
server_key_version: "encryptionkey@0"
I20260929 02:56:56.081385 26832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:56:56.084540 26847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:56.086086 26832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:56.086248 26832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "96aaf34e642e4db9be06df150807f696"
format_stamp: "Formatted at 2026-09-29 02:56:56 on dist-test-slave-1dkt"
server_key: "e7fb4578ac7d09ba38db7acd0fccc7cc"
server_key_iv: "6acc25a7f502eaa6fe6872c19209c72f"
server_key_version: "encryptionkey@0"
I20260929 02:56:56.086444 26832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:56.110195 26832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:56.114110 26832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:56.114354 26832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:56.115244 26832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:56.116544 26832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:56.116621 26832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:56.116736 26832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:56.116796 26832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:56.146054 26832 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:46259
I20260929 02:56:56.146092 26959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:46259 every 8 connection(s)
I20260929 02:56:56.147284 26832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:56:56.156162 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26832
I20260929 02:56:56.156317 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:56:56.156695 8420 external_mini_cluster.cc:1428] Setting key cdd16f528657239012f150e725e6ede6
Sep 29 02:56:56 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650616, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:56.166311 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.149841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:59457 (local address 127.8.57.62:41511)
0929 02:56:56.150015 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.150020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.151091 (+ 1071us) server_negotiation.cc:407] Connection header received
0929 02:56:56.152903 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.152909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.152989 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.153072 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:56.154505 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.155256 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.156854 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.157429 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.157526 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.161194 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:56.161198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:56.161201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:56.161232 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:56.163535 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.164235 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.164239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.164241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.164377 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.164753 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.164756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.164758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.165027 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:56.165155 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.165916 (+ 761us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.166085 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:56:56.167490 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.150237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41511 (local address 127.8.57.2:59457)
0929 02:56:56.150926 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.150967 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.152641 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.153301 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.153309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.153820 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:56.154344 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.154361 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.155405 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.155409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.156714 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.156727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.157647 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.157654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.157899 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.158537 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:56.158572 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:56.160991 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:56.163657 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:56.163665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:56.163680 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:56.164118 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:56.164476 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:56.164481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:56.164484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:56.164644 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:56.165145 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:56.165152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:56.165524 (+ 372us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.165809 (+ 285us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.166293 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":280,"threads_started":1}
I20260929 02:56:56.168872 26960 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41511
I20260929 02:56:56.169139 26960 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:56.169783 26960 heartbeater.cc:511] Master 127.8.57.62:41511 requested a full tablet report, sending...
I20260929 02:56:56.171168 26637 ts_manager.cc:194] Registered new tserver with Master: 96aaf34e642e4db9be06df150807f696 (127.8.57.2:46259)
I20260929 02:56:56.172204 26637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:59457
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:56:56.206439 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41511
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:56:56.513314 26965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:56.513587 26965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:56.513722 26965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:56.520684 26965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:56.520792 26965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:56.520953 26965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:56:56.530362 26965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:41511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:56.531559 26965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:56.532792 26965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:56.544795 26970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:56.544827 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:56.544862 26971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:56.546126 26965 server_base.cc:1034] running on GCE node
I20260929 02:56:56.546706 26965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:56.547587 26965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:56.548774 26965 hybrid_clock.cc:648] HybridClock initialized: now 1790650616548715 us; error 53 us; skew 500 ppm
I20260929 02:56:56.549074 26965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:56:56 dist-test-slave-1dkt krb5kdc[26592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650616, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:56:56.556627 26965 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:56:56.558068 26965 webserver.cc:460] Webserver started at http://127.8.57.3:42429/ using document root <none> and password file <none>
I20260929 02:56:56.558703 26965 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:56.558816 26965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:56.559095 26965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:56.561930 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "9128d896e6e641fa9154e06970713c1c"
format_stamp: "Formatted at 2026-09-29 02:56:56 on dist-test-slave-1dkt"
server_key: "696ca2b910b1a779671f27d4893b116f"
server_key_iv: "6a537cbb7e2f7042af1cdb03879f8cb8"
server_key_version: "encryptionkey@0"
I20260929 02:56:56.562700 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "9128d896e6e641fa9154e06970713c1c"
format_stamp: "Formatted at 2026-09-29 02:56:56 on dist-test-slave-1dkt"
server_key: "696ca2b910b1a779671f27d4893b116f"
server_key_iv: "6a537cbb7e2f7042af1cdb03879f8cb8"
server_key_version: "encryptionkey@0"
I20260929 02:56:56.567667 26965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 02:56:56.570940 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:56.572254 26965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:56.572399 26965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "9128d896e6e641fa9154e06970713c1c"
format_stamp: "Formatted at 2026-09-29 02:56:56 on dist-test-slave-1dkt"
server_key: "696ca2b910b1a779671f27d4893b116f"
server_key_iv: "6a537cbb7e2f7042af1cdb03879f8cb8"
server_key_version: "encryptionkey@0"
I20260929 02:56:56.572574 26965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:56.606518 26965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:56.610468 26965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:56.610725 26965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:56.611658 26965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:56.612955 26965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:56.613029 26965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:56.613154 26965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:56.613206 26965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:56.651474 26965 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:34103
I20260929 02:56:56.651525 27092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:34103 every 8 connection(s)
I20260929 02:56:56.652998 26965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:56:56.657483 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 26965
I20260929 02:56:56.657660 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:56:56.658042 8420 external_mini_cluster.cc:1428] Setting key 434688933a9b8d534d350dfea3113b45
Sep 29 02:56:56 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650616, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:56:56.675998 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.655727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:53607 (local address 127.8.57.62:41511)
0929 02:56:56.655960 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.655968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.657484 (+ 1516us) server_negotiation.cc:407] Connection header received
0929 02:56:56.660147 (+ 2663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.660153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.660234 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.660332 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:56.662350 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.663172 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.665537 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.665956 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.666062 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.670302 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:56.670307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:56.670310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:56.670344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:56.673435 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.674067 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.674070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.674072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.674190 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.674526 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.674529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.674531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.674765 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:56.674869 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.675521 (+ 652us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.675684 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:56:56.676932 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.656383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:41511 (local address 127.8.57.3:53607)
0929 02:56:56.657291 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.657342 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.659823 (+ 2481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.660612 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.660625 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.661356 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:56.662171 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.662197 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.663308 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.663314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.665375 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.665391 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.666177 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.666185 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.666485 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.667175 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:56:56.667211 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:56.670083 (+ 2872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:56.673561 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:56.673567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:56.673577 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:56.673976 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:56.674263 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:56:56.674266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:56:56.674268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:56:56.674448 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:56:56.674895 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:56.674901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:56.675189 (+ 288us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.675420 (+ 231us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.675809 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":345,"threads_started":1}
I20260929 02:56:56.678177 27093 heartbeater.cc:347] Connected to a master server at 127.8.57.62:41511
I20260929 02:56:56.678490 27093 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:56.679145 27093 heartbeater.cc:511] Master 127.8.57.62:41511 requested a full tablet report, sending...
I20260929 02:56:56.680462 26637 ts_manager.cc:194] Registered new tserver with Master: 9128d896e6e641fa9154e06970713c1c (127.8.57.3:34103)
I20260929 02:56:56.681218 26637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:53607
I20260929 02:56:56.685170 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:56:56.699179 26826 heartbeater.cc:503] Master 127.8.57.62:41511 was elected leader, sending a full tablet report...
I20260929 02:56:56.701936 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.689049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.8.57.62:41511)
0929 02:56:56.689218 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.689223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.689645 (+ 422us) server_negotiation.cc:407] Connection header received
0929 02:56:56.689759 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.689763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.689824 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.689920 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:56.691342 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.692134 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.693410 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.693863 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.693958 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.696447 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:56.696452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:56.696456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:56.696501 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:56.699042 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.700022 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.700027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.700029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.700156 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.700645 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.700650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.700653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.701033 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:56.701240 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.701555 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.701705 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:56:56.705714 26637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:56:56.712476 26637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:56:56.733500 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.725214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:34103 (local address 127.0.0.1:37452)
0929 02:56:56.725934 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.725949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.726048 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.727179 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.727183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.727206 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:56.727552 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.727563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.729455 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.729461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.731294 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.731308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.732727 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.732733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.732888 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.732927 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.732990 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.733103 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":46,"thread_start_us":649,"threads_started":1}
I20260929 02:56:56.734009 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.724070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.8.57.2:46259)
0929 02:56:56.724719 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.724729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.725282 (+ 553us) server_negotiation.cc:407] Connection header received
0929 02:56:56.725387 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.725394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.725650 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.725862 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:56.726849 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.728253 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.730588 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.731987 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.732108 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.732305 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.733539 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.733706 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":1263,"threads_started":1}
I20260929 02:56:56.734061 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.725698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.8.57.3:34103)
0929 02:56:56.726614 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.726622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.726663 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:56:56.726755 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.726805 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.727023 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.727172 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:56.727760 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.729287 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.731441 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.732501 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.733319 (+ 818us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.733532 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.733597 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.733731 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":1117,"threads_started":1}
I20260929 02:56:56.733235 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.723921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:46259 (local address 127.0.0.1:47678)
0929 02:56:56.724829 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.724871 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.725083 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.726142 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.726150 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.726183 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:56.726640 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.726658 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.728376 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.728381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.730439 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.730454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.732182 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.732188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.732430 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.732482 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.732599 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.732791 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":79,"thread_start_us":500,"threads_started":1}
I20260929 02:56:56.735265 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.724514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43813 (local address 127.0.0.1:37608)
0929 02:56:56.725306 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.725321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.725462 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.728397 (+ 2935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.728402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.728430 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:56.728827 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.728842 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.730773 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.730786 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.732802 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.732814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.734498 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.734504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.734663 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.734694 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.734772 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.734890 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":1,"thread_start_us":968,"threads_started":1}
I20260929 02:56:56.735303 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.725152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.8.57.1:43813)
0929 02:56:56.727839 (+ 2687us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.727846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.727906 (+ 60us) server_negotiation.cc:407] Connection header received
0929 02:56:56.727993 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.728000 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.728207 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.729015 (+ 808us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:56.729210 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.730650 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.732978 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.734326 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.734447 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.734648 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.734830 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.734997 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2514,"thread_start_us":2380,"threads_started":1}
I20260929 02:56:56.737812 27028 tablet_service.cc:1467] Processing CreateTablet for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 (DEFAULT_TABLE table=test-table [id=08e0c79f17e74fca84c2b2c55cfe0d50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:56.738965 27028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:56.738546 26761 tablet_service.cc:1467] Processing CreateTablet for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 (DEFAULT_TABLE table=test-table [id=08e0c79f17e74fca84c2b2c55cfe0d50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:56.738546 26895 tablet_service.cc:1467] Processing CreateTablet for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 (DEFAULT_TABLE table=test-table [id=08e0c79f17e74fca84c2b2c55cfe0d50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:56.739833 26895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:56.739846 26761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:56.750741 27111 tablet_bootstrap.cc:492] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Bootstrap starting.
I20260929 02:56:56.754082 27111 tablet_bootstrap.cc:654] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:56.754333 27112 tablet_bootstrap.cc:492] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Bootstrap starting.
I20260929 02:56:56.755928 27113 tablet_bootstrap.cc:492] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Bootstrap starting.
I20260929 02:56:56.756497 27111 log.cc:824] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:56.758917 27113 tablet_bootstrap.cc:654] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:56.758998 27112 tablet_bootstrap.cc:654] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:56.759915 27111 tablet_bootstrap.cc:492] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: No bootstrap required, opened a new log
I20260929 02:56:56.759915 27113 log.cc:824] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:56.760191 27111 ts_tablet_manager.cc:1397] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260929 02:56:56.760258 27112 log.cc:824] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:56.763257 27112 tablet_bootstrap.cc:492] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: No bootstrap required, opened a new log
I20260929 02:56:56.763525 27112 ts_tablet_manager.cc:1397] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 02:56:56.765923 27113 tablet_bootstrap.cc:492] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: No bootstrap required, opened a new log
I20260929 02:56:56.766167 27113 ts_tablet_manager.cc:1397] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260929 02:56:56.770159 27111 raft_consensus.cc:348] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.770597 27111 raft_consensus.cc:374] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:56.770702 27111 raft_consensus.cc:729] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9128d896e6e641fa9154e06970713c1c, State: Initialized, Role: FOLLOWER
I20260929 02:56:56.771332 27111 consensus_queue.cc:260] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.772424 27093 heartbeater.cc:503] Master 127.8.57.62:41511 was elected leader, sending a full tablet report...
I20260929 02:56:56.773134 27111 ts_tablet_manager.cc:1428] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260929 02:56:56.773586 27112 raft_consensus.cc:348] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.773964 27112 raft_consensus.cc:374] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:56.774067 27112 raft_consensus.cc:729] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2924d69a699e42eb996f860dd931fa41, State: Initialized, Role: FOLLOWER
I20260929 02:56:56.774676 27112 consensus_queue.cc:260] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.774710 27113 raft_consensus.cc:348] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.775027 27113 raft_consensus.cc:374] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:56.775112 27113 raft_consensus.cc:729] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96aaf34e642e4db9be06df150807f696, State: Initialized, Role: FOLLOWER
I20260929 02:56:56.775658 27113 consensus_queue.cc:260] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.778998 27113 ts_tablet_manager.cc:1428] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260929 02:56:56.779407 26960 heartbeater.cc:503] Master 127.8.57.62:41511 was elected leader, sending a full tablet report...
I20260929 02:56:56.780609 27112 ts_tablet_manager.cc:1428] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Time spent starting tablet: real 0.017s user 0.014s sys 0.005s
I20260929 02:56:56.890122 27117 raft_consensus.cc:482] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:56:56.890419 27117 raft_consensus.cc:504] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.892158 27117 leader_election.cc:288] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96aaf34e642e4db9be06df150807f696 (127.8.57.2:46259), 2924d69a699e42eb996f860dd931fa41 (127.8.57.1:43813)
I20260929 02:56:56.897505 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.892567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:43813 (local address 127.8.57.3:50415)
0929 02:56:56.892785 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.892820 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.892942 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.893264 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.893269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.893301 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:56.893475 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.893484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.894438 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.894443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.895721 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.895732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.896805 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.896810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.896980 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.897005 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.897086 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.897208 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 02:56:56.897552 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.892570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:50415 (local address 127.8.57.1:43813)
0929 02:56:56.892902 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.892908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.892928 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:56:56.893086 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.893091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.893152 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.893224 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:56.893606 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.894314 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.895893 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.896649 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.896766 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.896867 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.897121 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.897244 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260929 02:56:56.898061 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.892567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:60669 (local address 127.8.57.2:46259)
0929 02:56:56.892757 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.892763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.893390 (+ 627us) server_negotiation.cc:407] Connection header received
0929 02:56:56.893563 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.893567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.893624 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.893694 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:56.894150 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.894808 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.896441 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.897227 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.897342 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.897578 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.897765 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.897879 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:56:56.898576 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.892612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:46259 (local address 127.8.57.3:60669)
0929 02:56:56.893289 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:56.893302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:56.893440 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:56.893719 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:56.893723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:56.893746 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:56.894013 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.894021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.894941 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.894946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.896305 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:56.896316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.897431 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.897437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:56.897598 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.897628 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:56:56.898219 (+ 591us) client_negotiation.cc:240] Negotiation successful
0929 02:56:56.898329 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":389,"threads_started":1}
I20260929 02:56:56.898516 26781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0469a7f5c8a4b16a52ae0c4e4aa0ef3" candidate_uuid: "9128d896e6e641fa9154e06970713c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2924d69a699e42eb996f860dd931fa41" is_pre_election: true
I20260929 02:56:56.899068 26781 raft_consensus.cc:2454] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9128d896e6e641fa9154e06970713c1c in term 0.
W20260929 02:56:56.899298 26961 tablet.cc:2369] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:56.899507 26915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0469a7f5c8a4b16a52ae0c4e4aa0ef3" candidate_uuid: "9128d896e6e641fa9154e06970713c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96aaf34e642e4db9be06df150807f696" is_pre_election: true
I20260929 02:56:56.899813 26984 leader_election.cc:302] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2924d69a699e42eb996f860dd931fa41, 9128d896e6e641fa9154e06970713c1c; no voters:
I20260929 02:56:56.900059 26915 raft_consensus.cc:2454] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9128d896e6e641fa9154e06970713c1c in term 0.
I20260929 02:56:56.900238 27117 raft_consensus.cc:2781] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:56:56.900379 27117 raft_consensus.cc:482] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:56:56.900470 27117 raft_consensus.cc:3037] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:56.902575 27117 raft_consensus.cc:504] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.903100 27117 leader_election.cc:288] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [CANDIDATE]: Term 1 election: Requested vote from peers 96aaf34e642e4db9be06df150807f696 (127.8.57.2:46259), 2924d69a699e42eb996f860dd931fa41 (127.8.57.1:43813)
I20260929 02:56:56.903554 26915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0469a7f5c8a4b16a52ae0c4e4aa0ef3" candidate_uuid: "9128d896e6e641fa9154e06970713c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96aaf34e642e4db9be06df150807f696"
I20260929 02:56:56.903571 26781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0469a7f5c8a4b16a52ae0c4e4aa0ef3" candidate_uuid: "9128d896e6e641fa9154e06970713c1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2924d69a699e42eb996f860dd931fa41"
I20260929 02:56:56.903731 26915 raft_consensus.cc:3037] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:56.903748 26781 raft_consensus.cc:3037] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:56:56.904950 27094 tablet.cc:2369] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:56.906035 26781 raft_consensus.cc:2454] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9128d896e6e641fa9154e06970713c1c in term 1.
I20260929 02:56:56.906262 26915 raft_consensus.cc:2454] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9128d896e6e641fa9154e06970713c1c in term 1.
I20260929 02:56:56.906399 26984 leader_election.cc:302] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2924d69a699e42eb996f860dd931fa41, 9128d896e6e641fa9154e06970713c1c; no voters:
I20260929 02:56:56.906651 27117 raft_consensus.cc:2781] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:56.907080 27117 raft_consensus.cc:686] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 1 LEADER]: Becoming Leader. State: Replica: 9128d896e6e641fa9154e06970713c1c, State: Running, Role: LEADER
I20260929 02:56:56.907564 27117 consensus_queue.cc:237] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } }
I20260929 02:56:56.911916 26636 catalog_manager.cc:5495] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9128d896e6e641fa9154e06970713c1c (127.8.57.3). New cstate: current_term: 1 leader_uuid: "9128d896e6e641fa9154e06970713c1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9128d896e6e641fa9154e06970713c1c" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 } health_report { overall_health: UNKNOWN } } }
W20260929 02:56:56.921518 26827 tablet.cc:2369] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:56.927646 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.922420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37466 (local address 127.8.57.3:34103)
0929 02:56:56.922580 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.922584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.922829 (+ 245us) server_negotiation.cc:407] Connection header received
0929 02:56:56.922969 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.922973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.923059 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.923170 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:56:56.923501 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.924298 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.925686 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.926149 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.926248 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.926702 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:56:56.927198 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:56:56.927341 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.927419 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.927457 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:56:56.935048 26915 raft_consensus.cc:1261] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 1 FOLLOWER]: Refusing update from remote peer 9128d896e6e641fa9154e06970713c1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:56.935050 26781 raft_consensus.cc:1261] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 1 FOLLOWER]: Refusing update from remote peer 9128d896e6e641fa9154e06970713c1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:56.935956 27121 consensus_queue.cc:1035] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2924d69a699e42eb996f860dd931fa41" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:56.936281 27117 consensus_queue.cc:1035] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "96aaf34e642e4db9be06df150807f696" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:56.946300 27124 mvcc.cc:205] Tried to move back new op lower bound from 7334504926957285376 to 7334504926857424896. Current Snapshot: MvccSnapshot[applied={T|T < 7334504926957285376}]
I20260929 02:56:56.959149 26636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:56.959414 26636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:56.962668 26636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 766a9aefaeb74df191b5a4dd95e59d7c: Sending DeleteTablet for 3 replicas of tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3
I20260929 02:56:56.963704 27028 tablet_service.cc:1514] Processing DeleteTablet for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:56 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:37452
I20260929 02:56:56.963937 26895 tablet_service.cc:1514] Processing DeleteTablet for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:56 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:47678
I20260929 02:56:56.964411 26761 tablet_service.cc:1514] Processing DeleteTablet for tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:56 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:37608
I20260929 02:56:56.964740 27135 tablet_replica.cc:330] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: stopping tablet replica
I20260929 02:56:56.965139 27135 raft_consensus.cc:2229] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:56.965195 27136 tablet_replica.cc:330] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: stopping tablet replica
I20260929 02:56:56.965617 27135 raft_consensus.cc:2258] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:56.965641 27136 raft_consensus.cc:2229] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:56.966166 27136 raft_consensus.cc:2258] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:56.968050 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26698
I20260929 02:56:56.968089 27136 ts_tablet_manager.cc:1905] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:56.968277 27135 ts_tablet_manager.cc:1905] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:56.974934 27136 ts_tablet_manager.cc:1918] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:56.975157 27136 log.cc:1196] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/b0469a7f5c8a4b16a52ae0c4e4aa0ef3
I20260929 02:56:56.975329 27135 ts_tablet_manager.cc:1918] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:56.975481 27135 log.cc:1196] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/b0469a7f5c8a4b16a52ae0c4e4aa0ef3
I20260929 02:56:56.975939 27136 ts_tablet_manager.cc:1939] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 2924d69a699e42eb996f860dd931fa41: Deleting consensus metadata
I20260929 02:56:56.976011 27135 ts_tablet_manager.cc:1939] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 96aaf34e642e4db9be06df150807f696: Deleting consensus metadata
I20260929 02:56:56.977769 26624 catalog_manager.cc:4841] TS 2924d69a699e42eb996f860dd931fa41 (127.8.57.1:43813): tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 (table test-table [id=08e0c79f17e74fca84c2b2c55cfe0d50]) successfully deleted
I20260929 02:56:56.977818 26621 catalog_manager.cc:4841] TS 96aaf34e642e4db9be06df150807f696 (127.8.57.2:46259): tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 (table test-table [id=08e0c79f17e74fca84c2b2c55cfe0d50]) successfully deleted
I20260929 02:56:56.979028 27134 tablet_replica.cc:330] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: stopping tablet replica
I20260929 02:56:56.979457 27134 raft_consensus.cc:2229] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:56:56.980209 27134 raft_consensus.cc:2258] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:56.982249 27134 ts_tablet_manager.cc:1905] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:56.989351 27134 ts_tablet_manager.cc:1918] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:56.989507 27134 log.cc:1196] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/b0469a7f5c8a4b16a52ae0c4e4aa0ef3
Sep 29 02:56:56 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:56:56.990015 27134 ts_tablet_manager.cc:1939] T b0469a7f5c8a4b16a52ae0c4e4aa0ef3 P 9128d896e6e641fa9154e06970713c1c: Deleting consensus metadata
I20260929 02:56:56.991726 26621 catalog_manager.cc:4841] TS 9128d896e6e641fa9154e06970713c1c (127.8.57.3:34103): tablet b0469a7f5c8a4b16a52ae0c4e4aa0ef3 (table test-table [id=08e0c79f17e74fca84c2b2c55cfe0d50]) successfully deleted
I20260929 02:56:56.996865 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:56.969272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.8.57.1:43813)
0929 02:56:56.969440 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:56.969446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:56.978398 (+ 8952us) server_negotiation.cc:407] Connection header received
0929 02:56:56.978659 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:56.978666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:56.978751 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:56.978853 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:56.980368 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.981182 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.982473 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:56.982862 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:56.982969 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:56.991035 (+ 8066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:56.991050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:56.991068 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:56.991109 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:56.994216 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.994867 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.994876 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.994878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.995017 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:56.995392 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:56.995396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:56.995398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:56.995733 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:56.996042 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:56.996319 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 02:56:56.996536 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:56:56.997465 26821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:57.087767 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26698
I20260929 02:56:57.108985 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26832
Sep 29 02:56:57 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:56:57.123241 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:57.110041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.8.57.2:46259)
0929 02:56:57.110227 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:57.110231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:57.110246 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:56:57.110297 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:57.110301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:57.110367 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:57.110457 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:57.111637 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:57.112423 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:57.113655 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:57.114039 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:57.114141 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:57.118005 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:57.118017 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:57.118031 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:57.118074 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:57.120689 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:57.121351 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:57.121358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:57.121360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:57.121488 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:57.121829 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:57.121832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:57.121834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:57.122186 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:57.122451 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:57.122749 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 02:56:57.122949 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:56:57.124099 26955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:57.214006 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26832
I20260929 02:56:57.234051 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26965
Sep 29 02:56:57 dist-test-slave-1dkt krb5kdc[26592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:56:57.252192 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:57.235026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.8.57.3:34103)
0929 02:56:57.235220 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:57.235227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:57.235249 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:56:57.235364 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:57.235370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:57.235463 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:57.235590 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:57.237111 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:57.238369 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:57.239814 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:57.240598 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:57.240737 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:57.245136 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:57.245154 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:57.245174 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:56:57.245232 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:57.249158 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:57.249816 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:57.249825 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:57.249829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:57.250047 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:56:57.250416 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:56:57.250421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:56:57.250425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:56:57.250874 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:57.251219 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:57.251432 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 02:56:57.251647 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:56:57.253036 27088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:57.373320 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26965
I20260929 02:56:57.388141 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 26605
I20260929 02:56:57.389104 26667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:57.496695 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 26605
2026-09-29T02:56:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T02:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:56:57Z Disabled control of system clock
I20260929 02:56:57.562969 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:45407
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:45407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:56:57.868883 27145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:57.869154 27145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:57.869272 27145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:57.876336 27145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:56:57.876431 27145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:57.876492 27145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:57.876566 27145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:56:57.876621 27145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:56:57.885403 27145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:45407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:45407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:57.886582 27145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:57.888284 27145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:57.899312 27151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:57.899362 27150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:57.899386 27153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:57.900128 27145 server_base.cc:1034] running on GCE node
I20260929 02:56:57.900812 27145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:57.902060 27145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:57.903244 27145 hybrid_clock.cc:648] HybridClock initialized: now 1790650617903193 us; error 42 us; skew 500 ppm
I20260929 02:56:57.903544 27145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:57.905638 27145 webserver.cc:460] Webserver started at http://127.8.57.62:34443/ using document root <none> and password file <none>
I20260929 02:56:57.906251 27145 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:57.906345 27145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:57.906627 27145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:57.909355 27145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "5855205c7bc448f39627d48696a61a9a"
format_stamp: "Formatted at 2026-09-29 02:56:57 on dist-test-slave-1dkt"
server_key: "4af2e69f2ec5373be6f131af3cd0fd38"
server_key_iv: "5a0da0eebbd286bee010984fa0fe364e"
server_key_version: "encryptionkey@0"
I20260929 02:56:57.910130 27145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "5855205c7bc448f39627d48696a61a9a"
format_stamp: "Formatted at 2026-09-29 02:56:57 on dist-test-slave-1dkt"
server_key: "4af2e69f2ec5373be6f131af3cd0fd38"
server_key_iv: "5a0da0eebbd286bee010984fa0fe364e"
server_key_version: "encryptionkey@0"
I20260929 02:56:57.914783 27145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:56:57.917919 27159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:57.919090 27145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 02:56:57.919243 27145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "5855205c7bc448f39627d48696a61a9a"
format_stamp: "Formatted at 2026-09-29 02:56:57 on dist-test-slave-1dkt"
server_key: "4af2e69f2ec5373be6f131af3cd0fd38"
server_key_iv: "5a0da0eebbd286bee010984fa0fe364e"
server_key_version: "encryptionkey@0"
I20260929 02:56:57.919420 27145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:57.933862 27145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:57.934561 27145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:57.934796 27145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:57.951277 27145 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:45407
I20260929 02:56:57.951315 27210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:45407 every 8 connection(s)
I20260929 02:56:57.952502 27145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:56:57.955922 27211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:57.961015 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27145
I20260929 02:56:57.961158 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:56:57.961601 8420 external_mini_cluster.cc:1428] Setting key 60d8ccb504ef1d11ccdb1b8516fad712
I20260929 02:56:57.965904 27211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a: Bootstrap starting.
I20260929 02:56:57.969023 27211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:57.970173 27211 log.cc:824] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:57.973100 27211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a: No bootstrap required, opened a new log
I20260929 02:56:57.974859 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:57.963332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.8.57.62:45407)
0929 02:56:57.964314 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:57.964324 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:57.964362 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:56:57.965985 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:57.966004 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:57.966423 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:57.966717 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:57.967310 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:57.968609 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:57.970398 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:57.971093 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:57.971220 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:57.972276 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:57.972287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:57.972306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:57.972945 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:57.973039 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:57.973062 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:57.973175 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:57.973377 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 02:56:57.973703 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":339,"threads_started":1}
I20260929 02:56:57.981588 27211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5855205c7bc448f39627d48696a61a9a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45407 } }
I20260929 02:56:57.981890 27211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:57.981959 27211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5855205c7bc448f39627d48696a61a9a, State: Initialized, Role: FOLLOWER
I20260929 02:56:57.982535 27211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5855205c7bc448f39627d48696a61a9a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45407 } }
I20260929 02:56:57.982796 27211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:56:57.982926 27211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:56:57.983057 27211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:57.985064 27211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5855205c7bc448f39627d48696a61a9a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45407 } }
I20260929 02:56:57.985492 27211 leader_election.cc:302] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5855205c7bc448f39627d48696a61a9a; no voters:
I20260929 02:56:57.985970 27211 leader_election.cc:288] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:56:57.986100 27216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:57.986717 27216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [term 1 LEADER]: Becoming Leader. State: Replica: 5855205c7bc448f39627d48696a61a9a, State: Running, Role: LEADER
I20260929 02:56:57.987190 27216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5855205c7bc448f39627d48696a61a9a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45407 } }
I20260929 02:56:57.987807 27211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:56:57.989871 27218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5855205c7bc448f39627d48696a61a9a. Latest consensus state: current_term: 1 leader_uuid: "5855205c7bc448f39627d48696a61a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5855205c7bc448f39627d48696a61a9a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45407 } } }
I20260929 02:56:57.989847 27217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5855205c7bc448f39627d48696a61a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5855205c7bc448f39627d48696a61a9a" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 45407 } } }
I20260929 02:56:57.990319 27217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:57.990317 27218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a [sys.catalog]: This master's current role is: LEADER
I20260929 02:56:57.992231 27224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:56:57.997128 27224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:56:58.005259 27224 catalog_manager.cc:1347] Generated new cluster ID: 7c40621a4d0342e4a2b47922b24a1ceb
I20260929 02:56:58.005371 27224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:56:58.017982 27224 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:56:58.019698 27224 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:56:58.029043 27224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a: Generated new TSK 0
I20260929 02:56:58.029961 27224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:56:58.039106 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45407
--builtin_ntp_servers=127.8.57.20:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:56:58.366451 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:58.366704 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:58.366825 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:58.373808 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:58.373907 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:58.374051 27235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:56:58.382637 27235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:58.383762 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:58.385414 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:58.397241 27240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:58.397292 27243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:58.397401 27241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:58.398236 27235 server_base.cc:1034] running on GCE node
I20260929 02:56:58.398716 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:58.399506 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:58.400729 27235 hybrid_clock.cc:648] HybridClock initialized: now 1790650618400677 us; error 41 us; skew 500 ppm
I20260929 02:56:58.401027 27235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:58.403012 27235 webserver.cc:460] Webserver started at http://127.8.57.1:40275/ using document root <none> and password file <none>
I20260929 02:56:58.403612 27235 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:58.403705 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:58.404029 27235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:58.406651 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "a17f17728be048ec9e93db335da11116"
format_stamp: "Formatted at 2026-09-29 02:56:58 on dist-test-slave-1dkt"
server_key: "e75e4c1611a76f16bed2904385f13768"
server_key_iv: "274ac946a8918f1cefd73b78245b9c9a"
server_key_version: "encryptionkey@0"
I20260929 02:56:58.407374 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "a17f17728be048ec9e93db335da11116"
format_stamp: "Formatted at 2026-09-29 02:56:58 on dist-test-slave-1dkt"
server_key: "e75e4c1611a76f16bed2904385f13768"
server_key_iv: "274ac946a8918f1cefd73b78245b9c9a"
server_key_version: "encryptionkey@0"
I20260929 02:56:58.412113 27235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.007s
I20260929 02:56:58.415221 27249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:58.416455 27235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:56:58.416589 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "a17f17728be048ec9e93db335da11116"
format_stamp: "Formatted at 2026-09-29 02:56:58 on dist-test-slave-1dkt"
server_key: "e75e4c1611a76f16bed2904385f13768"
server_key_iv: "274ac946a8918f1cefd73b78245b9c9a"
server_key_version: "encryptionkey@0"
I20260929 02:56:58.416764 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:58.443001 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:58.443728 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:58.443993 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:58.444849 27235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:58.446230 27235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:58.446321 27235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:58.446432 27235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:58.446499 27235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:58.483062 27235 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:46281
I20260929 02:56:58.483125 27361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:46281 every 8 connection(s)
I20260929 02:56:58.484325 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:56:58.494359 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27235
I20260929 02:56:58.494511 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:56:58.495031 8420 external_mini_cluster.cc:1428] Setting key cd74663c3b8d453c94f8ba69afdb1d42
I20260929 02:56:58.496145 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:58.486663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:45331 (local address 127.8.57.62:45407)
0929 02:56:58.487080 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:58.487085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:58.488264 (+ 1179us) server_negotiation.cc:407] Connection header received
0929 02:56:58.490045 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:58.490051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:58.490125 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:58.490217 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:58.491192 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.492042 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.493490 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.493883 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.493988 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:58.495328 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:58.495335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:58.495338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:58.495354 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:58.495400 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:58.495412 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:58.495484 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:58.495683 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 02:56:58.495774 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":212,"threads_started":1}
I20260929 02:56:58.497084 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:58.487211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:45407 (local address 127.8.57.1:45331)
0929 02:56:58.488127 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:58.488161 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:58.489790 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:58.490435 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:58.490442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:58.490522 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:58.491039 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:58.491055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.492180 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.492183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:58.493359 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:58.493370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.494106 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.494114 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:58.494321 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:58.495034 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:56:58.495036 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:58.495085 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:56:58.495167 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:58.495496 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:58.495506 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:58.495514 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:56:58.495668 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 02:56:58.496053 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":272,"threads_started":1}
I20260929 02:56:58.497615 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45407
--builtin_ntp_servers=127.8.57.20:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:56:58.498486 27362 heartbeater.cc:347] Connected to a master server at 127.8.57.62:45407
I20260929 02:56:58.498740 27362 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:58.499377 27362 heartbeater.cc:511] Master 127.8.57.62:45407 requested a full tablet report, sending...
I20260929 02:56:58.501299 27176 ts_manager.cc:194] Registered new tserver with Master: a17f17728be048ec9e93db335da11116 (127.8.57.1:46281)
I20260929 02:56:58.502678 27176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.1:45331
W20260929 02:56:58.801865 27366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:58.802121 27366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:58.802244 27366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:58.809252 27366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:58.809355 27366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:58.809499 27366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:56:58.818202 27366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:58.819366 27366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:58.821054 27366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:58.833021 27371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:58.832985 27372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:58.833081 27374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:58.833666 27366 server_base.cc:1034] running on GCE node
I20260929 02:56:58.834182 27366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:58.835016 27366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:58.836195 27366 hybrid_clock.cc:648] HybridClock initialized: now 1790650618836144 us; error 43 us; skew 500 ppm
I20260929 02:56:58.836481 27366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:58.838581 27366 webserver.cc:460] Webserver started at http://127.8.57.2:39009/ using document root <none> and password file <none>
I20260929 02:56:58.839174 27366 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:58.839260 27366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:58.839547 27366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:58.842171 27366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "918a7eb01a3a456fb986057cfb317b79"
format_stamp: "Formatted at 2026-09-29 02:56:58 on dist-test-slave-1dkt"
server_key: "a45f829508bc0a275bad3c9998741611"
server_key_iv: "7cba1c6b38a9711ea71895aa7910e77f"
server_key_version: "encryptionkey@0"
I20260929 02:56:58.842890 27366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "918a7eb01a3a456fb986057cfb317b79"
format_stamp: "Formatted at 2026-09-29 02:56:58 on dist-test-slave-1dkt"
server_key: "a45f829508bc0a275bad3c9998741611"
server_key_iv: "7cba1c6b38a9711ea71895aa7910e77f"
server_key_version: "encryptionkey@0"
I20260929 02:56:58.847497 27366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260929 02:56:58.850821 27380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:58.852191 27366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:58.852335 27366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "918a7eb01a3a456fb986057cfb317b79"
format_stamp: "Formatted at 2026-09-29 02:56:58 on dist-test-slave-1dkt"
server_key: "a45f829508bc0a275bad3c9998741611"
server_key_iv: "7cba1c6b38a9711ea71895aa7910e77f"
server_key_version: "encryptionkey@0"
I20260929 02:56:58.852525 27366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:58.868933 27366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:58.869663 27366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:58.869920 27366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:58.870800 27366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:58.872223 27366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:58.872319 27366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:58.872421 27366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:58.872493 27366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:58.900785 27366 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:36191
I20260929 02:56:58.900830 27492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:36191 every 8 connection(s)
I20260929 02:56:58.902050 27366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:56:58.908578 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27366
I20260929 02:56:58.908721 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:56:58.909133 8420 external_mini_cluster.cc:1428] Setting key 8e75a8bf2296200d718716b3b25e3c3b
I20260929 02:56:58.911973 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45407
--builtin_ntp_servers=127.8.57.20:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:56:58.914868 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:58.904641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:49919 (local address 127.8.57.62:45407)
0929 02:56:58.904839 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:58.904877 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:58.906185 (+ 1308us) server_negotiation.cc:407] Connection header received
0929 02:56:58.907971 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:58.907976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:58.908062 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:58.908147 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:58.909275 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.910153 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.911834 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.912442 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.912571 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:58.914061 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:58.914068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:58.914072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:58.914095 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:58.914137 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:58.914153 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:58.914238 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:58.914469 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 02:56:58.914574 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:56:58.916062 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:58.905132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:45407 (local address 127.8.57.2:49919)
0929 02:56:58.906034 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:58.906069 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:58.907683 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:58.908385 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:58.908393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:58.908483 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:58.909097 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:58.909116 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.910316 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.910320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:58.911688 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:58.911701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:58.912671 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:58.912676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:58.912913 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:58.913668 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:56:58.913671 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:58.913715 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:56:58.913852 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:58.914268 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:58.914277 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:58.914284 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:56:58.914454 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 02:56:58.914896 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":265,"threads_started":1}
I20260929 02:56:58.917621 27493 heartbeater.cc:347] Connected to a master server at 127.8.57.62:45407
I20260929 02:56:58.917969 27493 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:58.918892 27493 heartbeater.cc:511] Master 127.8.57.62:45407 requested a full tablet report, sending...
I20260929 02:56:58.920483 27176 ts_manager.cc:194] Registered new tserver with Master: 918a7eb01a3a456fb986057cfb317b79 (127.8.57.2:36191)
I20260929 02:56:58.921630 27176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.2:49919
W20260929 02:56:59.236330 27496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:56:59.236594 27496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:56:59.236702 27496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:56:59.243975 27496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:56:59.244073 27496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:56:59.244215 27496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:56:59.253098 27496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:45407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:56:59.254266 27496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:56:59.256016 27496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:56:59.268306 27501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:59.268237 27504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:56:59.268275 27502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:56:59.268997 27496 server_base.cc:1034] running on GCE node
I20260929 02:56:59.269593 27496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:56:59.270498 27496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:56:59.271721 27496 hybrid_clock.cc:648] HybridClock initialized: now 1790650619271670 us; error 37 us; skew 500 ppm
I20260929 02:56:59.272086 27496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:56:59.274310 27496 webserver.cc:460] Webserver started at http://127.8.57.3:44365/ using document root <none> and password file <none>
I20260929 02:56:59.274977 27496 fs_manager.cc:359] Metadata directory not provided
I20260929 02:56:59.275066 27496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:56:59.275377 27496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:56:59.278483 27496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "a827458181fd4c88bf30f0b4ad8f7dab"
format_stamp: "Formatted at 2026-09-29 02:56:59 on dist-test-slave-1dkt"
server_key: "b6570117b15beb43f6a5e1122944598c"
server_key_iv: "9c0d33ed947fca61e5f7e054d5578404"
server_key_version: "encryptionkey@0"
I20260929 02:56:59.279230 27496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "a827458181fd4c88bf30f0b4ad8f7dab"
format_stamp: "Formatted at 2026-09-29 02:56:59 on dist-test-slave-1dkt"
server_key: "b6570117b15beb43f6a5e1122944598c"
server_key_iv: "9c0d33ed947fca61e5f7e054d5578404"
server_key_version: "encryptionkey@0"
I20260929 02:56:59.284214 27496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:56:59.287528 27510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:59.288888 27496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:56:59.289031 27496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "a827458181fd4c88bf30f0b4ad8f7dab"
format_stamp: "Formatted at 2026-09-29 02:56:59 on dist-test-slave-1dkt"
server_key: "b6570117b15beb43f6a5e1122944598c"
server_key_iv: "9c0d33ed947fca61e5f7e054d5578404"
server_key_version: "encryptionkey@0"
I20260929 02:56:59.289216 27496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:56:59.300424 27496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:56:59.301160 27496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:56:59.301407 27496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:56:59.302301 27496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:56:59.303689 27496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:56:59.303771 27496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:56:59.303894 27496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:56:59.303964 27496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 02:56:59.332088 27496 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:36195
I20260929 02:56:59.332129 27622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:36195 every 8 connection(s)
I20260929 02:56:59.333295 27496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:56:59.337158 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27496
I20260929 02:56:59.337322 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:56:59.337837 8420 external_mini_cluster.cc:1428] Setting key 9c7d2b3d9b71c169dc8fcb38036e73a6
I20260929 02:56:59.344921 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.335551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:47781 (local address 127.8.57.62:45407)
0929 02:56:59.335708 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.335712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.337037 (+ 1325us) server_negotiation.cc:407] Connection header received
0929 02:56:59.339194 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.339198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.339266 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.339384 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:59.340484 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.341185 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.342600 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.342988 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.343074 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.344289 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:59.344294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:59.344297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:59.344314 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:59.344346 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:59.344367 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:59.344438 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.344598 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.344688 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:56:59.345794 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.336021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:45407 (local address 127.8.57.3:47781)
0929 02:56:59.336872 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:59.336933 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:59.338944 (+ 2011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:59.339580 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:59.339590 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:59.339702 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:56:59.340345 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.340363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.341300 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.341304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.342493 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.342503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.343201 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.343206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.343390 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.344003 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:56:59.344006 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:56:59.344046 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:56:59.344116 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:56:59.344446 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:56:59.344452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:56:59.344457 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:56:59.344577 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 02:56:59.344827 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":246,"threads_started":1}
I20260929 02:56:59.347020 27623 heartbeater.cc:347] Connected to a master server at 127.8.57.62:45407
I20260929 02:56:59.347262 27623 heartbeater.cc:464] Registering TS with master...
I20260929 02:56:59.347944 27623 heartbeater.cc:511] Master 127.8.57.62:45407 requested a full tablet report, sending...
I20260929 02:56:59.349190 27176 ts_manager.cc:194] Registered new tserver with Master: a827458181fd4c88bf30f0b4ad8f7dab (127.8.57.3:36195)
I20260929 02:56:59.349951 27176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.3:47781
I20260929 02:56:59.352586 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:56:59.361503 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.356437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.8.57.62:45407)
0929 02:56:59.356689 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.356695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.356926 (+ 231us) server_negotiation.cc:407] Connection header received
0929 02:56:59.357036 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.357039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.357103 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.357178 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:59.357559 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.358212 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.359428 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.359823 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.359971 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.360888 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:59.360893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:59.360896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:59.360913 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:59.360943 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:59.360954 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:59.361056 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.361181 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.361226 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:56:59.364877 27176 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:56:59.371461 27176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:56:59.393795 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.382926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:36195 (local address 127.0.0.1:35632)
0929 02:56:59.384646 (+ 1720us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:59.384661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:59.384787 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:59.385717 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:59.385722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:59.385755 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:59.386339 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.386350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.388462 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.388467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.391276 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.391290 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.392711 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.392718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.392902 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.393009 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 02:56:59.393193 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 02:56:59.393445 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598,"mutex_wait_us":397,"thread_start_us":1450,"threads_started":1}
I20260929 02:56:59.394137 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.383237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.8.57.3:36195)
0929 02:56:59.384238 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.384246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.384903 (+ 657us) server_negotiation.cc:407] Connection header received
0929 02:56:59.385031 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.385040 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.385280 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.385460 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:59.386529 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.388297 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.391512 (+ 3215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.392537 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.392743 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.393019 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.393241 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.393413 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":1275,"threads_started":1}
I20260929 02:56:59.395366 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.382522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46281 (local address 127.0.0.1:51126)
0929 02:56:59.383479 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:59.383516 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:59.383670 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:59.385575 (+ 1905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:59.385584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:59.385617 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:59.386199 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.386213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.388437 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.388443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.391250 (+ 2807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.391266 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.394556 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.394564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.394763 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.394795 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:56:59.394898 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 02:56:59.395022 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":134,"thread_start_us":984,"threads_started":1}
I20260929 02:56:59.395453 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.382684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.8.57.1:46281)
0929 02:56:59.384933 (+ 2249us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.384941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.384970 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:56:59.385079 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.385087 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.385329 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.386429 (+ 1100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:59.386691 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.388276 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.391495 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.394365 (+ 2870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.394501 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.394712 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.394958 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.395116 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2090,"thread_start_us":2094,"threads_started":1}
I20260929 02:56:59.397511 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.384196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:36191 (local address 127.0.0.1:42996)
0929 02:56:59.389957 (+ 5761us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:59.389980 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:59.390098 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:59.390919 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:59.390922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:59.390940 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:59.391305 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.391318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.393129 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.393133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.395108 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.395122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.396809 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.396815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.396979 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.397009 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:56:59.397089 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:56:59.397212 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5645,"thread_start_us":5513,"threads_started":1}
I20260929 02:56:59.397596 27640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.384673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.8.57.2:36191)
0929 02:56:59.389573 (+ 4900us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.389582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.390216 (+ 634us) server_negotiation.cc:407] Connection header received
0929 02:56:59.390323 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.390330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.390552 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.390729 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:59.391491 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.392976 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.395322 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.396640 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.396755 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.397006 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.397131 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.397274 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4729,"thread_start_us":4600,"threads_started":1}
I20260929 02:56:59.398394 27558 tablet_service.cc:1467] Processing CreateTablet for tablet 4e106578b907400cba09edc6c1279bba (DEFAULT_TABLE table=test-table [id=61e18a55fe214858b056a888e8991e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:59.399118 27297 tablet_service.cc:1467] Processing CreateTablet for tablet 4e106578b907400cba09edc6c1279bba (DEFAULT_TABLE table=test-table [id=61e18a55fe214858b056a888e8991e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:59.399935 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e106578b907400cba09edc6c1279bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:59.400537 27297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e106578b907400cba09edc6c1279bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:59.401928 27428 tablet_service.cc:1467] Processing CreateTablet for tablet 4e106578b907400cba09edc6c1279bba (DEFAULT_TABLE table=test-table [id=61e18a55fe214858b056a888e8991e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:56:59.402951 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e106578b907400cba09edc6c1279bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:56:59.413390 27641 tablet_bootstrap.cc:492] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Bootstrap starting.
I20260929 02:56:59.416011 27642 tablet_bootstrap.cc:492] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Bootstrap starting.
I20260929 02:56:59.417788 27641 tablet_bootstrap.cc:654] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:59.419009 27641 log.cc:824] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:59.419278 27642 tablet_bootstrap.cc:654] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:59.420395 27642 log.cc:824] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:59.421080 27643 tablet_bootstrap.cc:492] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Bootstrap starting.
I20260929 02:56:59.422035 27641 tablet_bootstrap.cc:492] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: No bootstrap required, opened a new log
I20260929 02:56:59.422304 27641 ts_tablet_manager.cc:1397] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:56:59.423271 27642 tablet_bootstrap.cc:492] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: No bootstrap required, opened a new log
I20260929 02:56:59.423525 27642 ts_tablet_manager.cc:1397] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 02:56:59.424412 27643 tablet_bootstrap.cc:654] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Neither blocks nor log segments found. Creating new log.
I20260929 02:56:59.425546 27643 log.cc:824] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Log is configured to *not* fsync() on all Append() calls
I20260929 02:56:59.428393 27643 tablet_bootstrap.cc:492] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: No bootstrap required, opened a new log
I20260929 02:56:59.428632 27643 ts_tablet_manager.cc:1397] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:56:59.429781 27641 raft_consensus.cc:348] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.430102 27641 raft_consensus.cc:374] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:59.430190 27641 raft_consensus.cc:729] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918a7eb01a3a456fb986057cfb317b79, State: Initialized, Role: FOLLOWER
I20260929 02:56:59.430836 27641 consensus_queue.cc:260] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.432524 27493 heartbeater.cc:503] Master 127.8.57.62:45407 was elected leader, sending a full tablet report...
I20260929 02:56:59.433207 27641 ts_tablet_manager.cc:1428] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260929 02:56:59.433076 27642 raft_consensus.cc:348] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.433452 27642 raft_consensus.cc:374] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:59.433538 27642 raft_consensus.cc:729] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17f17728be048ec9e93db335da11116, State: Initialized, Role: FOLLOWER
I20260929 02:56:59.434091 27642 consensus_queue.cc:260] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.435604 27642 ts_tablet_manager.cc:1428] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260929 02:56:59.435935 27362 heartbeater.cc:503] Master 127.8.57.62:45407 was elected leader, sending a full tablet report...
I20260929 02:56:59.437968 27643 raft_consensus.cc:348] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.438361 27643 raft_consensus.cc:374] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:56:59.438463 27643 raft_consensus.cc:729] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a827458181fd4c88bf30f0b4ad8f7dab, State: Initialized, Role: FOLLOWER
I20260929 02:56:59.438897 27643 consensus_queue.cc:260] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.439766 27623 heartbeater.cc:503] Master 127.8.57.62:45407 was elected leader, sending a full tablet report...
I20260929 02:56:59.440119 27643 ts_tablet_manager.cc:1428] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260929 02:56:59.486420 27363 tablet.cc:2369] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:59.552181 27647 raft_consensus.cc:482] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:56:59.552466 27647 raft_consensus.cc:504] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.554220 27647 leader_election.cc:288] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a17f17728be048ec9e93db335da11116 (127.8.57.1:46281), a827458181fd4c88bf30f0b4ad8f7dab (127.8.57.3:36195)
I20260929 02:56:59.560652 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.554667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:48759 (local address 127.8.57.1:46281)
0929 02:56:59.554855 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.554861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.555506 (+ 645us) server_negotiation.cc:407] Connection header received
0929 02:56:59.555697 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.555701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.555769 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.555833 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:59.556449 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.557150 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.559078 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.559806 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.559943 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.560028 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.560285 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.560408 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:56:59.560658 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.554620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:51079 (local address 127.8.57.3:36195)
0929 02:56:59.554806 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.554812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.555498 (+ 686us) server_negotiation.cc:407] Connection header received
0929 02:56:59.555727 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.555731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.555790 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.555894 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:56:59.556580 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.557356 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.559088 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.559815 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.559971 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.560074 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.560313 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.560433 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:56:59.560782 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.554651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:36195 (local address 127.8.57.2:51079)
0929 02:56:59.555396 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:59.555412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:59.555567 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:59.555921 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:59.555925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:59.555947 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:59.556439 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.556451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.557485 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.557490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.558918 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.558929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.560018 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.560024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.560182 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.560204 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:56:59.560276 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:56:59.560400 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":22,"spinlock_wait_cycles":28160,"thread_start_us":443,"threads_started":1}
I20260929 02:56:59.561115 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.554592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:46281 (local address 127.8.57.2:48759)
0929 02:56:59.555397 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0929 02:56:59.555412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:56:59.555569 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:56:59.555894 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:56:59.555899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:56:59.555924 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:56:59.556311 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.556321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.557278 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.557284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.558893 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:56:59.558911 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.559962 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.559968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:56:59.560141 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.560163 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:56:59.560238 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 02:56:59.560355 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":65,"spinlock_wait_cycles":13056,"thread_start_us":453,"threads_started":1}
I20260929 02:56:59.561672 27578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e106578b907400cba09edc6c1279bba" candidate_uuid: "918a7eb01a3a456fb986057cfb317b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" is_pre_election: true
I20260929 02:56:59.562254 27578 raft_consensus.cc:2454] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918a7eb01a3a456fb986057cfb317b79 in term 0.
I20260929 02:56:59.562134 27317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e106578b907400cba09edc6c1279bba" candidate_uuid: "918a7eb01a3a456fb986057cfb317b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17f17728be048ec9e93db335da11116" is_pre_election: true
I20260929 02:56:59.562646 27317 raft_consensus.cc:2454] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918a7eb01a3a456fb986057cfb317b79 in term 0.
I20260929 02:56:59.562964 27383 leader_election.cc:302] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918a7eb01a3a456fb986057cfb317b79, a827458181fd4c88bf30f0b4ad8f7dab; no voters:
I20260929 02:56:59.563366 27647 raft_consensus.cc:2781] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:56:59.563482 27647 raft_consensus.cc:482] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:56:59.563571 27647 raft_consensus.cc:3037] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:59.565687 27647 raft_consensus.cc:504] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.566233 27647 leader_election.cc:288] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [CANDIDATE]: Term 1 election: Requested vote from peers a17f17728be048ec9e93db335da11116 (127.8.57.1:46281), a827458181fd4c88bf30f0b4ad8f7dab (127.8.57.3:36195)
I20260929 02:56:59.566634 27317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e106578b907400cba09edc6c1279bba" candidate_uuid: "918a7eb01a3a456fb986057cfb317b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17f17728be048ec9e93db335da11116"
I20260929 02:56:59.566690 27578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e106578b907400cba09edc6c1279bba" candidate_uuid: "918a7eb01a3a456fb986057cfb317b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a827458181fd4c88bf30f0b4ad8f7dab"
I20260929 02:56:59.566825 27317 raft_consensus.cc:3037] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:59.566854 27578 raft_consensus.cc:3037] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:56:59.569891 27578 raft_consensus.cc:2454] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918a7eb01a3a456fb986057cfb317b79 in term 1.
I20260929 02:56:59.569891 27317 raft_consensus.cc:2454] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918a7eb01a3a456fb986057cfb317b79 in term 1.
I20260929 02:56:59.570261 27384 leader_election.cc:302] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 918a7eb01a3a456fb986057cfb317b79, a17f17728be048ec9e93db335da11116; no voters:
I20260929 02:56:59.570513 27647 raft_consensus.cc:2781] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:56:59.570930 27647 raft_consensus.cc:686] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 1 LEADER]: Becoming Leader. State: Replica: 918a7eb01a3a456fb986057cfb317b79, State: Running, Role: LEADER
I20260929 02:56:59.571431 27647 consensus_queue.cc:237] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } }
I20260929 02:56:59.575973 27176 catalog_manager.cc:5495] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 918a7eb01a3a456fb986057cfb317b79 (127.8.57.2). New cstate: current_term: 1 leader_uuid: "918a7eb01a3a456fb986057cfb317b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "918a7eb01a3a456fb986057cfb317b79" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 36191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 } health_report { overall_health: UNKNOWN } } }
W20260929 02:56:59.584813 27624 tablet.cc:2369] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:56:59.592029 27640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.586564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.8.57.2:36191)
0929 02:56:59.586776 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.586782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.586906 (+ 124us) server_negotiation.cc:407] Connection header received
0929 02:56:59.587088 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.587093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.587178 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.587296 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:56:59.587680 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.588492 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.589890 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.590289 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.590395 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.590971 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:56:59.591449 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:56:59.591600 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.591750 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.591789 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:56:59.600003 27317 raft_consensus.cc:1261] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 1 FOLLOWER]: Refusing update from remote peer 918a7eb01a3a456fb986057cfb317b79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:59.600003 27578 raft_consensus.cc:1261] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 1 FOLLOWER]: Refusing update from remote peer 918a7eb01a3a456fb986057cfb317b79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:56:59.601043 27652 consensus_queue.cc:1035] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17f17728be048ec9e93db335da11116" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:59.601359 27647 consensus_queue.cc:1035] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a827458181fd4c88bf30f0b4ad8f7dab" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:56:59.625902 27176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:59.626123 27176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:56:59.629531 27176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5855205c7bc448f39627d48696a61a9a: Sending DeleteTablet for 3 replicas of tablet 4e106578b907400cba09edc6c1279bba
I20260929 02:56:59.630580 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e106578b907400cba09edc6c1279bba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:59 UTC) from {username='slave'} at 127.0.0.1:35632
I20260929 02:56:59.630704 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e106578b907400cba09edc6c1279bba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:59 UTC) from {username='slave'} at 127.0.0.1:42996
I20260929 02:56:59.631500 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27235
I20260929 02:56:59.631578 27297 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e106578b907400cba09edc6c1279bba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:56:59 UTC) from {username='slave'} at 127.0.0.1:51126
I20260929 02:56:59.636343 27667 tablet_replica.cc:330] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: stopping tablet replica
I20260929 02:56:59.636780 27667 raft_consensus.cc:2229] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:59.637298 27667 raft_consensus.cc:2258] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:59.641755 27667 ts_tablet_manager.cc:1905] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:59.643780 27666 tablet_replica.cc:330] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: stopping tablet replica
I20260929 02:56:59.644307 27666 raft_consensus.cc:2229] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:56:59.644989 27666 raft_consensus.cc:2258] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:59.646888 27666 ts_tablet_manager.cc:1905] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:59.648676 27667 ts_tablet_manager.cc:1918] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:59.648833 27667 log.cc:1196] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/4e106578b907400cba09edc6c1279bba
I20260929 02:56:59.649375 27667 ts_tablet_manager.cc:1939] T 4e106578b907400cba09edc6c1279bba P a17f17728be048ec9e93db335da11116: Deleting consensus metadata
I20260929 02:56:59.651173 27163 catalog_manager.cc:4841] TS a17f17728be048ec9e93db335da11116 (127.8.57.1:46281): tablet 4e106578b907400cba09edc6c1279bba (table test-table [id=61e18a55fe214858b056a888e8991e8e]) successfully deleted
I20260929 02:56:59.651978 27665 tablet_replica.cc:330] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: stopping tablet replica
I20260929 02:56:59.652456 27665 raft_consensus.cc:2229] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:56:59.652698 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.632801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51134 (local address 127.8.57.1:46281)
0929 02:56:59.632993 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.633000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.643935 (+ 10935us) server_negotiation.cc:407] Connection header received
0929 02:56:59.644416 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.644423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.644530 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.644604 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:59.645315 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.646579 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.648647 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.649424 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.649523 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.651021 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:59.651033 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:59.651052 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:59.651806 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:59.651851 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:59.651941 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:59.652024 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.652260 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.652365 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:56:59.653195 27665 raft_consensus.cc:2258] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:56:59.653728 27357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:59.654664 27666 ts_tablet_manager.cc:1918] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:59.654800 27666 log.cc:1196] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/4e106578b907400cba09edc6c1279bba
I20260929 02:56:59.655230 27666 ts_tablet_manager.cc:1939] T 4e106578b907400cba09edc6c1279bba P 918a7eb01a3a456fb986057cfb317b79: Deleting consensus metadata
I20260929 02:56:59.656672 27163 catalog_manager.cc:4841] TS 918a7eb01a3a456fb986057cfb317b79 (127.8.57.2:36191): tablet 4e106578b907400cba09edc6c1279bba (table test-table [id=61e18a55fe214858b056a888e8991e8e]) successfully deleted
I20260929 02:56:59.662086 27665 ts_tablet_manager.cc:1905] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:56:59.669485 27665 ts_tablet_manager.cc:1918] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:56:59.669662 27665 log.cc:1196] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/4e106578b907400cba09edc6c1279bba
I20260929 02:56:59.670167 27665 ts_tablet_manager.cc:1939] T 4e106578b907400cba09edc6c1279bba P a827458181fd4c88bf30f0b4ad8f7dab: Deleting consensus metadata
I20260929 02:56:59.672034 27162 catalog_manager.cc:4841] TS a827458181fd4c88bf30f0b4ad8f7dab (127.8.57.3:36195): tablet 4e106578b907400cba09edc6c1279bba (table test-table [id=61e18a55fe214858b056a888e8991e8e]) successfully deleted
I20260929 02:56:59.751564 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27235
I20260929 02:56:59.772598 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27366
I20260929 02:56:59.780308 27640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.773388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.8.57.2:36191)
0929 02:56:59.773590 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.773596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.773617 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:56:59.773690 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.773696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.773799 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.773910 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:59.774275 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.775443 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.776828 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.777463 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.777589 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.778641 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:59.778651 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:59.778670 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:59.779458 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:59.779531 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:59.779556 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:59.779664 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.779810 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.779930 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:56:59.781140 27488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:59.889539 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27366
I20260929 02:56:59.904373 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27496
I20260929 02:56:59.910916 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:56:59.905107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.8.57.3:36195)
0929 02:56:59.905272 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:56:59.905278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:56:59.905298 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:56:59.905491 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:56:59.905495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:56:59.905583 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:56:59.905668 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:56:59.906077 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.906792 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.908088 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:56:59.908488 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:56:59.908597 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:56:59.909645 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:56:59.909655 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:56:59.909668 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:56:59.910214 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:56:59.910265 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:56:59.910285 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:56:59.910392 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:56:59.910516 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 02:56:59.910621 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:56:59.911577 27618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:56:59.998272 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27496
I20260929 02:57:00.010892 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27145
I20260929 02:57:00.012034 27206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:00.119800 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27145
2026-09-29T02:57:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2621 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:00 dist-test-slave-1dkt krb5kdc[27675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:00 dist-test-slave-1dkt krb5kdc[27675](info): set up 2 sockets
Sep 29 02:57:00 dist-test-slave-1dkt krb5kdc[27675](info): commencing operation
krb5kdc: starting...
W20260929 02:57:01.136967 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:01 dist-test-slave-1dkt krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:01.362264 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42671
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:01.673842 27688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:01.674166 27688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:01.674311 27688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:01.681437 27688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:01.681546 27688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:01.681623 27688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:01.681699 27688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:01.681757 27688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:01.691370 27688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:01.692581 27688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:01.693817 27688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:01.704528 27693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:01.704641 27694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:01.704609 27696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:01.705597 27688 server_base.cc:1034] running on GCE node
I20260929 02:57:01.706247 27688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:01.707487 27688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:01.708674 27688 hybrid_clock.cc:648] HybridClock initialized: now 1790650621708635 us; error 29 us; skew 500 ppm
I20260929 02:57:01.708951 27688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:01 dist-test-slave-1dkt krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:01.711817 27688 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:01.713152 27688 webserver.cc:460] Webserver started at http://127.8.57.62:44489/ using document root <none> and password file <none>
I20260929 02:57:01.713771 27688 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:01.713877 27688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:01.714143 27688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:01.716958 27688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "bccead2891494c488a411d9b3211171d"
format_stamp: "Formatted at 2026-09-29 02:57:01 on dist-test-slave-1dkt"
server_key: "fdb552a37592f1120b8dfccb9ee015dc"
server_key_iv: "b0317d1fa8345ce40ef54d12717a7459"
server_key_version: "encryptionkey@0"
I20260929 02:57:01.717664 27688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "bccead2891494c488a411d9b3211171d"
format_stamp: "Formatted at 2026-09-29 02:57:01 on dist-test-slave-1dkt"
server_key: "fdb552a37592f1120b8dfccb9ee015dc"
server_key_iv: "b0317d1fa8345ce40ef54d12717a7459"
server_key_version: "encryptionkey@0"
I20260929 02:57:01.722615 27688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:57:01.726001 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:01.727304 27688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:01.727479 27688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "bccead2891494c488a411d9b3211171d"
format_stamp: "Formatted at 2026-09-29 02:57:01 on dist-test-slave-1dkt"
server_key: "fdb552a37592f1120b8dfccb9ee015dc"
server_key_iv: "b0317d1fa8345ce40ef54d12717a7459"
server_key_version: "encryptionkey@0"
I20260929 02:57:01.727659 27688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:01.753798 27688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:01.757797 27688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:01.758052 27688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:01.774832 27688 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42671
I20260929 02:57:01.774863 27754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42671 every 8 connection(s)
I20260929 02:57:01.776156 27688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:01.779760 27755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:01.782780 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27688
I20260929 02:57:01.782967 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:01.783491 8420 external_mini_cluster.cc:1428] Setting key d79f78895fb8db3821a7d6e1b4ca3ff6
I20260929 02:57:01.791536 27755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d: Bootstrap starting.
I20260929 02:57:01.794994 27755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:01.796229 27755 log.cc:824] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:57:01 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:01.798899 27755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d: No bootstrap required, opened a new log
I20260929 02:57:01.805516 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:01.785334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.8.57.62:42671)
0929 02:57:01.786077 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:01.786088 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:01.786128 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:57:01.787767 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:01.787788 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:01.788253 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:01.788566 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:01.790222 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:01.791497 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:01.793254 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:01.793920 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:01.794040 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:01.798640 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:01.798647 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:01.798660 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:01.798694 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:01.801730 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:01.802365 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:01.802371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:01.802377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:01.802535 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:01.802895 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:01.802899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:01.802901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:01.803438 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:01.803725 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:01.804146 (+ 421us) server_negotiation.cc:299] Negotiation successful
0929 02:57:01.804540 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":343,"threads_started":1}
I20260929 02:57:01.806550 27755 raft_consensus.cc:348] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccead2891494c488a411d9b3211171d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42671 } }
I20260929 02:57:01.806880 27755 raft_consensus.cc:374] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:01.806972 27755 raft_consensus.cc:729] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bccead2891494c488a411d9b3211171d, State: Initialized, Role: FOLLOWER
I20260929 02:57:01.807533 27755 consensus_queue.cc:260] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccead2891494c488a411d9b3211171d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42671 } }
I20260929 02:57:01.807770 27755 raft_consensus.cc:388] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:01.807971 27755 raft_consensus.cc:482] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:01.808147 27755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:01.810130 27755 raft_consensus.cc:504] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccead2891494c488a411d9b3211171d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42671 } }
I20260929 02:57:01.810570 27755 leader_election.cc:302] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bccead2891494c488a411d9b3211171d; no voters:
I20260929 02:57:01.811065 27755 leader_election.cc:288] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:01.811194 27760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:01.812218 27760 raft_consensus.cc:686] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [term 1 LEADER]: Becoming Leader. State: Replica: bccead2891494c488a411d9b3211171d, State: Running, Role: LEADER
I20260929 02:57:01.812769 27760 consensus_queue.cc:237] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccead2891494c488a411d9b3211171d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42671 } }
I20260929 02:57:01.812953 27755 sys_catalog.cc:564] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:01.815644 27761 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bccead2891494c488a411d9b3211171d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccead2891494c488a411d9b3211171d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42671 } } }
I20260929 02:57:01.816085 27761 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:01.817384 27769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:01.815671 27762 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bccead2891494c488a411d9b3211171d. Latest consensus state: current_term: 1 leader_uuid: "bccead2891494c488a411d9b3211171d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccead2891494c488a411d9b3211171d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42671 } } }
I20260929 02:57:01.817623 27762 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:01.821798 27769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:01.829797 27769 catalog_manager.cc:1347] Generated new cluster ID: a02947997b634957af80585f26beca16
I20260929 02:57:01.829903 27769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:01.842048 27769 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:01.843822 27769 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:01.853929 27769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d: Generated new TSK 0
I20260929 02:57:01.854864 27769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.57.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
Entry for principal kudu/127.8.57.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab.
I20260929 02:57:01.910635 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42671
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:02.225979 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:02.226240 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:02.226362 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:02.233664 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:02.233758 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:02.233912 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:57:02.243216 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.1.keytab
--principal=kudu/127.8.57.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:02.244427 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:02.245677 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:02.257966 27787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:02.258253 27786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:02.258225 27789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:02.258747 27781 server_base.cc:1034] running on GCE node
I20260929 02:57:02.259313 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:02.260263 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:02.261461 27781 hybrid_clock.cc:648] HybridClock initialized: now 1790650622261413 us; error 38 us; skew 500 ppm
I20260929 02:57:02.261752 27781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:02 dist-test-slave-1dkt krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:02.267853 27781 init.cc:377] Logged in from keytab as kudu/127.8.57.1@KRBTEST.COM (short username kudu)
I20260929 02:57:02.269284 27781 webserver.cc:460] Webserver started at http://127.8.57.1:40387/ using document root <none> and password file <none>
I20260929 02:57:02.269936 27781 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:02.270052 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:02.270339 27781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:02.273213 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "8d658d8712024f82ac12c25d9e0f4cb3"
format_stamp: "Formatted at 2026-09-29 02:57:02 on dist-test-slave-1dkt"
server_key: "ab82fbe7290fed702fa057f633cf7d4d"
server_key_iv: "450319c4dbab4f0a084ecbc893f54af4"
server_key_version: "encryptionkey@0"
I20260929 02:57:02.273959 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "8d658d8712024f82ac12c25d9e0f4cb3"
format_stamp: "Formatted at 2026-09-29 02:57:02 on dist-test-slave-1dkt"
server_key: "ab82fbe7290fed702fa057f633cf7d4d"
server_key_iv: "450319c4dbab4f0a084ecbc893f54af4"
server_key_version: "encryptionkey@0"
I20260929 02:57:02.278867 27781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:57:02.282195 27796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:02.283442 27781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:02.283584 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "8d658d8712024f82ac12c25d9e0f4cb3"
format_stamp: "Formatted at 2026-09-29 02:57:02 on dist-test-slave-1dkt"
server_key: "ab82fbe7290fed702fa057f633cf7d4d"
server_key_iv: "450319c4dbab4f0a084ecbc893f54af4"
server_key_version: "encryptionkey@0"
I20260929 02:57:02.283756 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:02.306802 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:02.310772 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:02.311038 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:02.311956 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:02.313251 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:02.313333 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:02.313460 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:02.313511 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:02.341781 27781 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:45777
I20260929 02:57:02.341863 27908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:45777 every 8 connection(s)
I20260929 02:57:02.342978 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:57:02.352473 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27781
I20260929 02:57:02.352612 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:57:02.353056 8420 external_mini_cluster.cc:1428] Setting key 81a8d1cd0325c75a058a7ddc19e55767
Sep 29 02:57:02 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.1@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:02.361863 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:02.345301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:46791 (local address 127.8.57.62:42671)
0929 02:57:02.345727 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:02.345731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:02.346550 (+ 819us) server_negotiation.cc:407] Connection header received
0929 02:57:02.348383 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:02.348389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:02.348467 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:02.348571 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:02.349993 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.350733 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.352260 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.352691 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.352803 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:02.356237 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:02.356242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:02.356247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:02.356306 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:02.359101 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:02.359757 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:02.359762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:02.359765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:02.359925 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:02.360275 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:02.360279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:02.360282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:02.360610 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:02.360751 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:02.361314 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 02:57:02.361513 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":226,"threads_started":1}
I20260929 02:57:02.362608 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:02.345728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42671 (local address 127.8.57.1:46791)
0929 02:57:02.346394 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:02.346430 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:02.348127 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:02.348797 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:02.348805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:02.349275 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:02.349831 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:02.349849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.350879 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.350882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:02.352132 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:02.352144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.352897 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.352905 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:02.353169 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:02.353856 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:02.353884 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:02.356019 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:02.359233 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:02.359240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:02.359251 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:02.359648 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:02.360010 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:02.360013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:02.360015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:02.360180 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:02.360712 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:02.360717 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:02.360995 (+ 278us) client_negotiation.cc:769] Sending connection context
0929 02:57:02.361199 (+ 204us) client_negotiation.cc:240] Negotiation successful
0929 02:57:02.361567 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":249,"threads_started":1}
I20260929 02:57:02.364209 27909 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42671
I20260929 02:57:02.364477 27909 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:02.365127 27909 heartbeater.cc:511] Master 127.8.57.62:42671 requested a full tablet report, sending...
I20260929 02:57:02.367054 27720 ts_manager.cc:194] Registered new tserver with Master: 8d658d8712024f82ac12c25d9e0f4cb3 (127.8.57.1:45777)
I20260929 02:57:02.368508 27720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.1@KRBTEST.COM'} at 127.8.57.1:46791
WARNING: no policy specified for kudu/127.8.57.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
Entry for principal kudu/127.8.57.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab.
I20260929 02:57:02.405223 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42671
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:02.709837 27915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:02.710106 27915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:02.710222 27915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:02.717581 27915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:02.717691 27915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:02.717833 27915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:57:02.727607 27915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.2.keytab
--principal=kudu/127.8.57.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:02.728893 27915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:02.730114 27915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:02.743014 27920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:02.743010 27923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:02.743083 27921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:02.743727 27915 server_base.cc:1034] running on GCE node
I20260929 02:57:02.744361 27915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:02.745354 27915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:02.746555 27915 hybrid_clock.cc:648] HybridClock initialized: now 1790650622746488 us; error 65 us; skew 500 ppm
I20260929 02:57:02.746827 27915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:02 dist-test-slave-1dkt krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:02.749842 27915 init.cc:377] Logged in from keytab as kudu/127.8.57.2@KRBTEST.COM (short username kudu)
I20260929 02:57:02.751231 27915 webserver.cc:460] Webserver started at http://127.8.57.2:42765/ using document root <none> and password file <none>
I20260929 02:57:02.751863 27915 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:02.752005 27915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:02.752264 27915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:02.755067 27915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "01f9e36f5e204304b99670dccade1c8d"
format_stamp: "Formatted at 2026-09-29 02:57:02 on dist-test-slave-1dkt"
server_key: "a68bf34228f53139413c214bd1e7b9fd"
server_key_iv: "500284abfd849c95cb1935b3eb1503e3"
server_key_version: "encryptionkey@0"
I20260929 02:57:02.755834 27915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "01f9e36f5e204304b99670dccade1c8d"
format_stamp: "Formatted at 2026-09-29 02:57:02 on dist-test-slave-1dkt"
server_key: "a68bf34228f53139413c214bd1e7b9fd"
server_key_iv: "500284abfd849c95cb1935b3eb1503e3"
server_key_version: "encryptionkey@0"
I20260929 02:57:02.760898 27915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:57:02.764145 27930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:02.765417 27915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:57:02.765590 27915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "01f9e36f5e204304b99670dccade1c8d"
format_stamp: "Formatted at 2026-09-29 02:57:02 on dist-test-slave-1dkt"
server_key: "a68bf34228f53139413c214bd1e7b9fd"
server_key_iv: "500284abfd849c95cb1935b3eb1503e3"
server_key_version: "encryptionkey@0"
I20260929 02:57:02.765790 27915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:02.798146 27915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:02.801884 27915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:02.802136 27915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:02.803085 27915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:02.804404 27915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:02.804487 27915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:02.804600 27915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:02.804651 27915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:02.834395 27915 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:37193
I20260929 02:57:02.834463 28042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:37193 every 8 connection(s)
I20260929 02:57:02.835598 27915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:57:02.837010 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 27915
I20260929 02:57:02.837178 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:57:02.837801 8420 external_mini_cluster.cc:1428] Setting key 8ca1d96802df1b136b160b61fbcd93d7
Sep 29 02:57:02 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.2@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:02.856537 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:02.838306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:50725 (local address 127.8.57.62:42671)
0929 02:57:02.838522 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:02.838529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:02.839752 (+ 1223us) server_negotiation.cc:407] Connection header received
0929 02:57:02.841583 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:02.841588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:02.841666 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:02.841763 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:02.843113 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.844078 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.845594 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.846137 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.846249 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:02.850211 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:02.850217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:02.850222 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:02.850264 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:02.853279 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:02.854217 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:02.854223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:02.854226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:02.854413 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:02.854838 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:02.854843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:02.854846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:02.855192 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:02.855337 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:02.856008 (+ 671us) server_negotiation.cc:299] Negotiation successful
0929 02:57:02.856234 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:57:02.857410 28045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:02.838823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42671 (local address 127.8.57.2:50725)
0929 02:57:02.839591 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:02.839643 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:02.841319 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:02.841938 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:02.841945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:02.842427 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:02.842959 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:02.842976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.844219 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.844223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:02.845456 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:02.845468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:02.846340 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:02.846346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:02.846564 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:02.847241 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:02.847272 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:02.849975 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:02.853425 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:02.853432 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:02.853445 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:02.854068 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:02.854528 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:02.854531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:02.854534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:02.854726 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:02.855305 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:02.855310 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:02.855646 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 02:57:02.855866 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 02:57:02.856323 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":278,"threads_started":1}
I20260929 02:57:02.858814 28043 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42671
I20260929 02:57:02.859153 28043 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:02.860044 28043 heartbeater.cc:511] Master 127.8.57.62:42671 requested a full tablet report, sending...
I20260929 02:57:02.861594 27720 ts_manager.cc:194] Registered new tserver with Master: 01f9e36f5e204304b99670dccade1c8d (127.8.57.2:37193)
I20260929 02:57:02.862344 27720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.2@KRBTEST.COM'} at 127.8.57.2:50725
WARNING: no policy specified for kudu/127.8.57.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
Entry for principal kudu/127.8.57.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab.
I20260929 02:57:02.891716 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42671
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:03.204794 28048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:03.205062 28048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:03.205180 28048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:03.212230 28048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:03.212323 28048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:03.212461 28048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:57:03.222049 28048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.3.keytab
--principal=kudu/127.8.57.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:42671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:03.223255 28048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:03.224565 28048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:03.237152 28054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:03.237217 28056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:03.237412 28053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:03.238103 28048 server_base.cc:1034] running on GCE node
I20260929 02:57:03.238694 28048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:03.239682 28048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:03.240904 28048 hybrid_clock.cc:648] HybridClock initialized: now 1790650623240870 us; error 88 us; skew 500 ppm
I20260929 02:57:03.241235 28048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:03 dist-test-slave-1dkt krb5kdc[27675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:03.244326 28048 init.cc:377] Logged in from keytab as kudu/127.8.57.3@KRBTEST.COM (short username kudu)
I20260929 02:57:03.245808 28048 webserver.cc:460] Webserver started at http://127.8.57.3:37601/ using document root <none> and password file <none>
I20260929 02:57:03.246457 28048 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:03.246552 28048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:03.246803 28048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:03.250294 28048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b"
format_stamp: "Formatted at 2026-09-29 02:57:03 on dist-test-slave-1dkt"
server_key: "e6a3f06e5a462aca800801422dcbe71a"
server_key_iv: "626151efc15257728b4d0e22f259f21d"
server_key_version: "encryptionkey@0"
I20260929 02:57:03.251063 28048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b"
format_stamp: "Formatted at 2026-09-29 02:57:03 on dist-test-slave-1dkt"
server_key: "e6a3f06e5a462aca800801422dcbe71a"
server_key_iv: "626151efc15257728b4d0e22f259f21d"
server_key_version: "encryptionkey@0"
I20260929 02:57:03.257426 28048 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260929 02:57:03.262188 28063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:03.263644 28048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:03.263803 28048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b"
format_stamp: "Formatted at 2026-09-29 02:57:03 on dist-test-slave-1dkt"
server_key: "e6a3f06e5a462aca800801422dcbe71a"
server_key_iv: "626151efc15257728b4d0e22f259f21d"
server_key_version: "encryptionkey@0"
I20260929 02:57:03.264053 28048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:03.279175 28048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:03.283422 28048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:03.283689 28048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:03.284758 28048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:03.286062 28048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:03.286144 28048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:03.286340 28048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:03.286422 28048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:03.317237 28048 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:34487
I20260929 02:57:03.317319 28175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:34487 every 8 connection(s)
I20260929 02:57:03.318435 28048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:57:03.323220 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28048
I20260929 02:57:03.323421 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:57:03.324026 8420 external_mini_cluster.cc:1428] Setting key cc89da44706c00e0aa222b6807e1cd30
Sep 29 02:57:03 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.3@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:03.337769 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.320910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:55075 (local address 127.8.57.62:42671)
0929 02:57:03.321087 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.321092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.322220 (+ 1128us) server_negotiation.cc:407] Connection header received
0929 02:57:03.324556 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.324562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.324646 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.324741 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:03.326498 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.327376 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.328913 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.329335 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.329430 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.332690 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:03.332694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:03.332697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:03.332729 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:03.334965 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.335603 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.335607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.335609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.335722 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.336150 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.336154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.336156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.336421 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:03.336558 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.337196 (+ 638us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.337417 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:57:03.338713 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.321352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:42671 (local address 127.8.57.3:55075)
0929 02:57:03.322064 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:03.322112 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:03.324256 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:03.324992 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:03.325002 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:03.325671 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:03.326323 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.326341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.327516 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.327520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.328790 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.328802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.329524 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.329529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.329711 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.330316 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:03.330337 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:03.332504 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:03.335091 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:03.335099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:03.335113 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:03.335486 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:03.335819 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:03.335824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:03.335828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:03.336044 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:03.336544 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:03.336551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:03.336849 (+ 298us) client_negotiation.cc:769] Sending connection context
0929 02:57:03.337086 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 02:57:03.337556 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":267,"threads_started":1}
I20260929 02:57:03.340008 28176 heartbeater.cc:347] Connected to a master server at 127.8.57.62:42671
I20260929 02:57:03.340288 28176 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:03.340956 28176 heartbeater.cc:511] Master 127.8.57.62:42671 requested a full tablet report, sending...
I20260929 02:57:03.342412 27720 ts_manager.cc:194] Registered new tserver with Master: 4be1b05d8bbb4da4a3ca4ce78c81214b (127.8.57.3:34487)
I20260929 02:57:03.343187 27720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.57.3@KRBTEST.COM'} at 127.8.57.3:55075
I20260929 02:57:03.351111 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:57:03.367005 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.355170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.8.57.62:42671)
0929 02:57:03.355313 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.355317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.355762 (+ 445us) server_negotiation.cc:407] Connection header received
0929 02:57:03.355974 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.355978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.356048 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.356120 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:03.357396 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.358104 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.359398 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.359797 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.359924 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.362244 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:03.362249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:03.362252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:03.362283 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:03.364535 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.365181 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.365185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.365188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.365299 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.365619 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.365624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.365627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.365917 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:03.366136 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.366483 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.366716 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:57:03.371126 27909 heartbeater.cc:503] Master 127.8.57.62:42671 was elected leader, sending a full tablet report...
I20260929 02:57:03.371008 27720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:57:03.378068 27720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:57:03.399369 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.389505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49410 (local address 127.8.57.2:37193)
0929 02:57:03.391896 (+ 2391us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.391913 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.391942 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:57:03.392046 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.392054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.392340 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.393195 (+ 855us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:03.393438 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.394654 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.397160 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.398219 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.398346 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.398573 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.398854 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.399039 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2243,"thread_start_us":2170,"threads_started":1}
I20260929 02:57:03.399482 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.389421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:37193 (local address 127.0.0.1:49410)
0929 02:57:03.391242 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:03.391283 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:03.391439 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:03.392559 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:03.392565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:03.392591 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:03.393007 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.393021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.394792 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.394797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.397018 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.397030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.398393 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.398399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.398606 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.398672 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 02:57:03.398810 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 02:57:03.399043 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1601,"mutex_wait_us":29,"thread_start_us":2597,"threads_started":1}
I20260929 02:57:03.400635 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.390004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.8.57.3:34487)
0929 02:57:03.390722 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.390732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.392382 (+ 1650us) server_negotiation.cc:407] Connection header received
0929 02:57:03.392670 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.392679 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.392903 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.393066 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:03.394049 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.395502 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.397942 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.399262 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.399417 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.399982 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.400182 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.400336 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":356,"threads_started":1}
I20260929 02:57:03.401178 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.389445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:34487 (local address 127.0.0.1:38374)
0929 02:57:03.392279 (+ 2834us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:03.392296 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:03.392500 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:03.393359 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:03.393363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:03.393386 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:03.393837 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.393854 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.395623 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.395628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.397775 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.397789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.399768 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.399774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.400027 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.400052 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 02:57:03.400131 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 02:57:03.400254 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2701,"thread_start_us":1430,"threads_started":1}
I20260929 02:57:03.403903 28192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.392358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:45777 (local address 127.0.0.1:37244)
0929 02:57:03.393632 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:03.393650 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:03.393781 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:03.396065 (+ 2284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:03.396070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:03.396099 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:03.396528 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.396539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.398248 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.398254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.401425 (+ 3171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.401439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.403045 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.403052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.403227 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.403261 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:57:03.403341 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:57:03.403499 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"mutex_wait_us":1,"thread_start_us":1001,"threads_started":1}
I20260929 02:57:03.403867 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.392925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37244 (local address 127.8.57.1:45777)
0929 02:57:03.395228 (+ 2303us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.395235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.395260 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:57:03.395355 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.395362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.395576 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.395769 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:03.396716 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.398128 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.401596 (+ 3468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.402831 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.402968 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.403191 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.403422 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.403573 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2145,"thread_start_us":2005,"threads_started":1}
I20260929 02:57:03.403435 27978 tablet_service.cc:1467] Processing CreateTablet for tablet a6d54c9b5543438db12214edb8c0a6d3 (DEFAULT_TABLE table=test-table [id=c534d07854a14d00becebea9555d6173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:03.404979 27978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d54c9b5543438db12214edb8c0a6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:03.405071 28111 tablet_service.cc:1467] Processing CreateTablet for tablet a6d54c9b5543438db12214edb8c0a6d3 (DEFAULT_TABLE table=test-table [id=c534d07854a14d00becebea9555d6173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:03.406471 28111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d54c9b5543438db12214edb8c0a6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:03.410553 27844 tablet_service.cc:1467] Processing CreateTablet for tablet a6d54c9b5543438db12214edb8c0a6d3 (DEFAULT_TABLE table=test-table [id=c534d07854a14d00becebea9555d6173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:03.412068 27844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d54c9b5543438db12214edb8c0a6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:03.420243 28194 tablet_bootstrap.cc:492] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Bootstrap starting.
I20260929 02:57:03.424032 28194 tablet_bootstrap.cc:654] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:03.424608 28196 tablet_bootstrap.cc:492] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Bootstrap starting.
I20260929 02:57:03.425520 28194 log.cc:824] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:03.426614 28195 tablet_bootstrap.cc:492] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Bootstrap starting.
I20260929 02:57:03.427994 28196 tablet_bootstrap.cc:654] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:03.428982 28196 log.cc:824] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:03.429464 28195 tablet_bootstrap.cc:654] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:03.430362 28195 log.cc:824] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:03.431586 28196 tablet_bootstrap.cc:492] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: No bootstrap required, opened a new log
I20260929 02:57:03.431815 28196 ts_tablet_manager.cc:1397] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:57:03.433151 28194 tablet_bootstrap.cc:492] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: No bootstrap required, opened a new log
I20260929 02:57:03.433388 28194 ts_tablet_manager.cc:1397] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260929 02:57:03.433532 28195 tablet_bootstrap.cc:492] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: No bootstrap required, opened a new log
I20260929 02:57:03.433760 28195 ts_tablet_manager.cc:1397] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260929 02:57:03.440537 28194 raft_consensus.cc:348] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.440886 28194 raft_consensus.cc:374] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:03.441015 28194 raft_consensus.cc:729] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01f9e36f5e204304b99670dccade1c8d, State: Initialized, Role: FOLLOWER
I20260929 02:57:03.441598 28194 consensus_queue.cc:260] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.441799 28196 raft_consensus.cc:348] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.442186 28196 raft_consensus.cc:374] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:03.442278 28196 raft_consensus.cc:729] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d658d8712024f82ac12c25d9e0f4cb3, State: Initialized, Role: FOLLOWER
I20260929 02:57:03.442744 28043 heartbeater.cc:503] Master 127.8.57.62:42671 was elected leader, sending a full tablet report...
I20260929 02:57:03.442898 28196 consensus_queue.cc:260] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.443145 28194 ts_tablet_manager.cc:1428] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260929 02:57:03.444640 28196 ts_tablet_manager.cc:1428] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:57:03.444384 28195 raft_consensus.cc:348] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.444964 28195 raft_consensus.cc:374] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:03.445071 28195 raft_consensus.cc:729] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be1b05d8bbb4da4a3ca4ce78c81214b, State: Initialized, Role: FOLLOWER
I20260929 02:57:03.445587 28195 consensus_queue.cc:260] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.446523 28176 heartbeater.cc:503] Master 127.8.57.62:42671 was elected leader, sending a full tablet report...
I20260929 02:57:03.447111 28195 ts_tablet_manager.cc:1428] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260929 02:57:03.569993 28177 tablet.cc:2369] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:57:03.587561 28044 tablet.cc:2369] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:57:03.595239 27910 tablet.cc:2369] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:57:03.715662 28200 raft_consensus.cc:482] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:57:03.716032 28200 raft_consensus.cc:504] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.717693 28200 leader_election.cc:288] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4be1b05d8bbb4da4a3ca4ce78c81214b (127.8.57.3:34487), 8d658d8712024f82ac12c25d9e0f4cb3 (127.8.57.1:45777)
I20260929 02:57:03.724031 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.718250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54683 (local address 127.8.57.1:45777)
0929 02:57:03.718462 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.718467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.718845 (+ 378us) server_negotiation.cc:407] Connection header received
0929 02:57:03.719031 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.719037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.719115 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.719190 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:03.719800 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.720572 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.722303 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.723086 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.723209 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.723309 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.723624 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.723751 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:57:03.724046 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.718162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:60025 (local address 127.8.57.3:34487)
0929 02:57:03.718383 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.718390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.719010 (+ 620us) server_negotiation.cc:407] Connection header received
0929 02:57:03.719156 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.719161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.719235 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.719331 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:03.719936 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.720658 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.722358 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.723102 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.723208 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.723308 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.723675 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.723797 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:57:03.724305 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.718039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:45777 (local address 127.8.57.2:54683)
0929 02:57:03.718703 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:03.718726 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:03.718883 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:03.719218 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:03.719222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:03.719245 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:03.719646 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.719659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.720694 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.720699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.722156 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.722167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.723313 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.723321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.723484 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.723508 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:57:03.723575 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:57:03.723684 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":1,"thread_start_us":345,"threads_started":1}
I20260929 02:57:03.724341 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.718095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:34487 (local address 127.8.57.2:60025)
0929 02:57:03.718904 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:03.718919 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:03.719046 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:03.719358 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:03.719361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:03.719393 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:03.719763 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.719776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.720783 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.720789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.722215 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:03.722229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.723351 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.723356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:03.723523 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.723545 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:57:03.723633 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 02:57:03.723749 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":22,"thread_start_us":506,"threads_started":1}
I20260929 02:57:03.725296 28131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d54c9b5543438db12214edb8c0a6d3" candidate_uuid: "01f9e36f5e204304b99670dccade1c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" is_pre_election: true
I20260929 02:57:03.725318 27864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d54c9b5543438db12214edb8c0a6d3" candidate_uuid: "01f9e36f5e204304b99670dccade1c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" is_pre_election: true
I20260929 02:57:03.725858 28131 raft_consensus.cc:2454] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01f9e36f5e204304b99670dccade1c8d in term 0.
I20260929 02:57:03.725858 27864 raft_consensus.cc:2454] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01f9e36f5e204304b99670dccade1c8d in term 0.
I20260929 02:57:03.726528 27932 leader_election.cc:302] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01f9e36f5e204304b99670dccade1c8d, 4be1b05d8bbb4da4a3ca4ce78c81214b; no voters:
I20260929 02:57:03.726905 28200 raft_consensus.cc:2781] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:57:03.727049 28200 raft_consensus.cc:482] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:57:03.727155 28200 raft_consensus.cc:3037] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:03.729272 28200 raft_consensus.cc:504] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.729888 28200 leader_election.cc:288] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [CANDIDATE]: Term 1 election: Requested vote from peers 4be1b05d8bbb4da4a3ca4ce78c81214b (127.8.57.3:34487), 8d658d8712024f82ac12c25d9e0f4cb3 (127.8.57.1:45777)
I20260929 02:57:03.730216 28131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d54c9b5543438db12214edb8c0a6d3" candidate_uuid: "01f9e36f5e204304b99670dccade1c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b"
I20260929 02:57:03.730409 28131 raft_consensus.cc:3037] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:03.730525 27864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d54c9b5543438db12214edb8c0a6d3" candidate_uuid: "01f9e36f5e204304b99670dccade1c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d658d8712024f82ac12c25d9e0f4cb3"
I20260929 02:57:03.730711 27864 raft_consensus.cc:3037] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:03.733505 28131 raft_consensus.cc:2454] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01f9e36f5e204304b99670dccade1c8d in term 1.
I20260929 02:57:03.733667 27864 raft_consensus.cc:2454] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01f9e36f5e204304b99670dccade1c8d in term 1.
I20260929 02:57:03.733978 27932 leader_election.cc:302] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01f9e36f5e204304b99670dccade1c8d, 4be1b05d8bbb4da4a3ca4ce78c81214b; no voters:
I20260929 02:57:03.734251 28200 raft_consensus.cc:2781] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:03.734683 28200 raft_consensus.cc:686] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 1 LEADER]: Becoming Leader. State: Replica: 01f9e36f5e204304b99670dccade1c8d, State: Running, Role: LEADER
I20260929 02:57:03.735106 28200 consensus_queue.cc:237] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } }
I20260929 02:57:03.739605 27718 catalog_manager.cc:5495] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 01f9e36f5e204304b99670dccade1c8d (127.8.57.2). New cstate: current_term: 1 leader_uuid: "01f9e36f5e204304b99670dccade1c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9e36f5e204304b99670dccade1c8d" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 37193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 } health_report { overall_health: UNKNOWN } } }
I20260929 02:57:03.764434 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.759456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49420 (local address 127.8.57.2:37193)
0929 02:57:03.759636 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.759640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.759657 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:57:03.759739 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.759743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.759823 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.759965 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:03.760270 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.760973 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.762257 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.762655 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.762746 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.763143 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:03.763648 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:03.763749 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.763846 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.763956 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:57:03.772485 27864 raft_consensus.cc:1261] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 1 FOLLOWER]: Refusing update from remote peer 01f9e36f5e204304b99670dccade1c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:03.773427 28131 raft_consensus.cc:1261] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 1 FOLLOWER]: Refusing update from remote peer 01f9e36f5e204304b99670dccade1c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:03.773509 28205 consensus_queue.cc:1035] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d658d8712024f82ac12c25d9e0f4cb3" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:03.774151 28200 consensus_queue.cc:1035] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4be1b05d8bbb4da4a3ca4ce78c81214b" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:03.786932 28208 mvcc.cc:205] Tried to move back new op lower bound from 7334504954962264064 to 7334504954822537216. Current Snapshot: MvccSnapshot[applied={T|T < 7334504954962264064}]
I20260929 02:57:03.807642 27720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:57:03.807986 27720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:57:03.811205 27720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bccead2891494c488a411d9b3211171d: Sending DeleteTablet for 3 replicas of tablet a6d54c9b5543438db12214edb8c0a6d3
I20260929 02:57:03.812536 28111 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d54c9b5543438db12214edb8c0a6d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:57:03 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:38374
I20260929 02:57:03.812531 27844 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d54c9b5543438db12214edb8c0a6d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:57:03 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:37244
I20260929 02:57:03.813267 27978 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d54c9b5543438db12214edb8c0a6d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:57:03 UTC) from {username='kudu', principal='kudu/127.8.57.62@KRBTEST.COM'} at 127.0.0.1:49410
I20260929 02:57:03.813413 28218 tablet_replica.cc:330] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: stopping tablet replica
I20260929 02:57:03.813999 28218 raft_consensus.cc:2229] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:57:03.814548 28218 raft_consensus.cc:2258] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:57:03.815258 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27781
I20260929 02:57:03.815316 28217 tablet_replica.cc:330] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: stopping tablet replica
I20260929 02:57:03.815806 28217 raft_consensus.cc:2229] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:57:03.816406 28217 raft_consensus.cc:2258] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:57:03.818550 28217 ts_tablet_manager.cc:1905] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:57:03.820427 28218 ts_tablet_manager.cc:1905] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:57:03.821363 28219 tablet_replica.cc:330] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: stopping tablet replica
I20260929 02:57:03.821787 28219 raft_consensus.cc:2229] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:57:03.822433 28219 raft_consensus.cc:2258] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:57:03.826980 28217 ts_tablet_manager.cc:1918] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:57:03.826980 28218 ts_tablet_manager.cc:1918] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:57:03.827160 28218 log.cc:1196] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/a6d54c9b5543438db12214edb8c0a6d3
I20260929 02:57:03.827164 28217 log.cc:1196] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/a6d54c9b5543438db12214edb8c0a6d3
I20260929 02:57:03.827694 28217 ts_tablet_manager.cc:1939] T a6d54c9b5543438db12214edb8c0a6d3 P 8d658d8712024f82ac12c25d9e0f4cb3: Deleting consensus metadata
I20260929 02:57:03.827693 28218 ts_tablet_manager.cc:1939] T a6d54c9b5543438db12214edb8c0a6d3 P 4be1b05d8bbb4da4a3ca4ce78c81214b: Deleting consensus metadata
I20260929 02:57:03.829404 28219 ts_tablet_manager.cc:1905] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:57:03.829561 27704 catalog_manager.cc:4841] TS 8d658d8712024f82ac12c25d9e0f4cb3 (127.8.57.1:45777): tablet a6d54c9b5543438db12214edb8c0a6d3 (table test-table [id=c534d07854a14d00becebea9555d6173]) successfully deleted
I20260929 02:57:03.830039 27705 catalog_manager.cc:4841] TS 4be1b05d8bbb4da4a3ca4ce78c81214b (127.8.57.3:34487): tablet a6d54c9b5543438db12214edb8c0a6d3 (table test-table [id=c534d07854a14d00becebea9555d6173]) successfully deleted
Sep 29 02:57:03 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.1@KRBTEST.COM
I20260929 02:57:03.835722 28219 ts_tablet_manager.cc:1918] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:57:03.835919 28219 log.cc:1196] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/a6d54c9b5543438db12214edb8c0a6d3
I20260929 02:57:03.836485 28219 ts_tablet_manager.cc:1939] T a6d54c9b5543438db12214edb8c0a6d3 P 01f9e36f5e204304b99670dccade1c8d: Deleting consensus metadata
I20260929 02:57:03.837889 27704 catalog_manager.cc:4841] TS 01f9e36f5e204304b99670dccade1c8d (127.8.57.2:37193): tablet a6d54c9b5543438db12214edb8c0a6d3 (table test-table [id=c534d07854a14d00becebea9555d6173]) successfully deleted
I20260929 02:57:03.842311 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.816804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.8.57.1:45777)
0929 02:57:03.816991 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.816998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.817225 (+ 227us) server_negotiation.cc:407] Connection header received
0929 02:57:03.820201 (+ 2976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.820207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.820301 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.820374 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:03.822351 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.823483 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.825475 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.826157 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.826258 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.831742 (+ 5484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:03.831758 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:03.831776 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:03.831828 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:03.839518 (+ 7690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.840283 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.840291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.840293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.840430 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.840814 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.840818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.840820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.841168 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:03.841456 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.841742 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.841959 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:57:03.842912 27904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:03.931327 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27781
I20260929 02:57:03.945495 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27915
Sep 29 02:57:03 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.2@KRBTEST.COM
I20260929 02:57:03.963321 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:03.946297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49424 (local address 127.8.57.2:37193)
0929 02:57:03.946492 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:03.946499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:03.946760 (+ 261us) server_negotiation.cc:407] Connection header received
0929 02:57:03.946955 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:03.946960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:03.947032 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:03.947119 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:03.948765 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.949526 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.951468 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:03.952073 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:03.952170 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:03.957627 (+ 5457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:03.957642 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:03.957657 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:03.957697 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:03.960667 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.961446 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.961454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.961456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.961597 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:03.961999 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:03.962002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:03.962004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:03.962330 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:03.962582 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:03.962849 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 02:57:03.963019 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:57:03.964061 28038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:04.075958 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27915
I20260929 02:57:04.090788 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28048
Sep 29 02:57:04 dist-test-slave-1dkt krb5kdc[27675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.3@KRBTEST.COM
I20260929 02:57:04.105350 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:04.091522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.8.57.3:34487)
0929 02:57:04.091700 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:04.091706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:04.091726 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:57:04.091948 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:04.091952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:04.092045 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:04.092131 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:04.093467 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:04.094179 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:04.095455 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:04.095848 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:04.096008 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:04.100255 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:04.100271 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:04.100287 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:04.100328 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:04.102706 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:04.103336 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:04.103344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:04.103346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:04.103489 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:04.103843 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:04.103847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:04.103849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:04.104324 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:04.104591 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:04.104816 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 02:57:04.105016 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:57:04.105997 28171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:04.194658 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28048
I20260929 02:57:04.207384 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 27688
I20260929 02:57:04.208493 27750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:04.318324 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 27688
2026-09-29T02:57:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4199 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T02:57:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:04Z Disabled control of system clock
I20260929 02:57:04.398954 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:34165
--webserver_interface=127.8.57.62
--webserver_port=0
--builtin_ntp_servers=127.8.57.20:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:34165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:57:04.722563 28228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:04.722843 28228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:04.722944 28228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:04.730209 28228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:04.730329 28228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:04.730366 28228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:04.730425 28228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:04.730469 28228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:04.739677 28228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:34165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:34165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:04.740978 28228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:04.742648 28228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:04.754068 28234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:04.754196 28233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:04.754035 28236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:04.754828 28228 server_base.cc:1034] running on GCE node
I20260929 02:57:04.755611 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:04.757085 28228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:04.758302 28228 hybrid_clock.cc:648] HybridClock initialized: now 1790650624758247 us; error 48 us; skew 500 ppm
I20260929 02:57:04.758637 28228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:57:04.760941 28228 webserver.cc:460] Webserver started at http://127.8.57.62:33947/ using document root <none> and password file <none>
I20260929 02:57:04.761720 28228 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:04.761833 28228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:04.762195 28228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:04.765205 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "1f16bcb69bb54ba6828ce5b47fdb4177"
format_stamp: "Formatted at 2026-09-29 02:57:04 on dist-test-slave-1dkt"
server_key: "c467e102d528f0942c2e45119b0495ed"
server_key_iv: "bc994313c7cef93f989102332fd9e796"
server_key_version: "encryptionkey@0"
I20260929 02:57:04.765983 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "1f16bcb69bb54ba6828ce5b47fdb4177"
format_stamp: "Formatted at 2026-09-29 02:57:04 on dist-test-slave-1dkt"
server_key: "c467e102d528f0942c2e45119b0495ed"
server_key_iv: "bc994313c7cef93f989102332fd9e796"
server_key_version: "encryptionkey@0"
I20260929 02:57:04.771119 28228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:57:04.774654 28242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:04.775995 28228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:04.776152 28228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "1f16bcb69bb54ba6828ce5b47fdb4177"
format_stamp: "Formatted at 2026-09-29 02:57:04 on dist-test-slave-1dkt"
server_key: "c467e102d528f0942c2e45119b0495ed"
server_key_iv: "bc994313c7cef93f989102332fd9e796"
server_key_version: "encryptionkey@0"
I20260929 02:57:04.776345 28228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:04.792975 28228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:04.793838 28228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:04.794108 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:04.811233 28228 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:34165
I20260929 02:57:04.811257 28293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:34165 every 8 connection(s)
I20260929 02:57:04.812546 28228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:04.813938 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28228
I20260929 02:57:04.814113 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:04.814702 8420 external_mini_cluster.cc:1428] Setting key ee4dcb28ff02dabe06046f3bb12ebfc7
I20260929 02:57:04.817122 28294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:04.827965 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:04.816588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.8.57.62:34165)
0929 02:57:04.817607 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:04.817620 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:04.817667 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:57:04.819312 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:04.819336 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:04.819852 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:04.820315 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:04.820908 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:04.822230 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:04.823615 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:04.824334 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:04.824524 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:04.825625 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:04.825633 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:04.825648 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:04.826229 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:04.826308 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:04.826331 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:04.826441 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:04.826660 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 02:57:04.826968 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":355,"threads_started":1}
I20260929 02:57:04.828082 28294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177: Bootstrap starting.
I20260929 02:57:04.831357 28294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:04.832381 28294 log.cc:824] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:04.835088 28294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177: No bootstrap required, opened a new log
I20260929 02:57:04.842474 28294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34165 } }
I20260929 02:57:04.842828 28294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:04.842931 28294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f16bcb69bb54ba6828ce5b47fdb4177, State: Initialized, Role: FOLLOWER
I20260929 02:57:04.843493 28294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34165 } }
I20260929 02:57:04.843713 28294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:04.843834 28294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:04.844035 28294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:04.846082 28294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34165 } }
I20260929 02:57:04.846542 28294 leader_election.cc:302] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f16bcb69bb54ba6828ce5b47fdb4177; no voters:
I20260929 02:57:04.847115 28294 leader_election.cc:288] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:04.847262 28299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:04.848428 28299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [term 1 LEADER]: Becoming Leader. State: Replica: 1f16bcb69bb54ba6828ce5b47fdb4177, State: Running, Role: LEADER
I20260929 02:57:04.848989 28299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34165 } }
I20260929 02:57:04.849300 28294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:04.851702 28300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34165 } } }
I20260929 02:57:04.852177 28300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:04.851751 28301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f16bcb69bb54ba6828ce5b47fdb4177. Latest consensus state: current_term: 1 leader_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f16bcb69bb54ba6828ce5b47fdb4177" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34165 } } }
I20260929 02:57:04.852883 28301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:04.854022 28307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:04.858523 28307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:04.866695 28307 catalog_manager.cc:1347] Generated new cluster ID: 79668fb1413641bbb4293b252b8fbefb
I20260929 02:57:04.866816 28307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:04.883297 28307 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:04.885190 28307 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:04.892376 28307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177: Generated new TSK 0
I20260929 02:57:04.893502 28307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:04.899940 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.1:0
--local_ip_for_outbound_sockets=127.8.57.1
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34165
--builtin_ntp_servers=127.8.57.20:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:57:05.232054 28318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:05.232350 28318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:05.232460 28318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:05.239711 28318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:05.239811 28318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:05.239972 28318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.1
I20260929 02:57:05.249105 28318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.57.1
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.1
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:05.250401 28318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:05.252178 28318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:05.264959 28323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:05.264990 28324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:05.264920 28326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:05.265758 28318 server_base.cc:1034] running on GCE node
I20260929 02:57:05.266307 28318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:05.267236 28318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:05.268476 28318 hybrid_clock.cc:648] HybridClock initialized: now 1790650625268424 us; error 40 us; skew 500 ppm
I20260929 02:57:05.268782 28318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:57:05.271126 28318 webserver.cc:460] Webserver started at http://127.8.57.1:33043/ using document root <none> and password file <none>
I20260929 02:57:05.271806 28318 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:05.271942 28318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:05.272272 28318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:05.275348 28318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data/instance:
uuid: "5665a13f9de64ec5b4bda85d173ccdda"
format_stamp: "Formatted at 2026-09-29 02:57:05 on dist-test-slave-1dkt"
server_key: "491c9ee2a9dbb6ad4e71f9707d848b55"
server_key_iv: "652ba7870d6291119c95cbd641f3fdf9"
server_key_version: "encryptionkey@0"
I20260929 02:57:05.276222 28318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance:
uuid: "5665a13f9de64ec5b4bda85d173ccdda"
format_stamp: "Formatted at 2026-09-29 02:57:05 on dist-test-slave-1dkt"
server_key: "491c9ee2a9dbb6ad4e71f9707d848b55"
server_key_iv: "652ba7870d6291119c95cbd641f3fdf9"
server_key_version: "encryptionkey@0"
I20260929 02:57:05.281556 28318 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:57:05.285157 28332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:05.286509 28318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:57:05.286660 28318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal
uuid: "5665a13f9de64ec5b4bda85d173ccdda"
format_stamp: "Formatted at 2026-09-29 02:57:05 on dist-test-slave-1dkt"
server_key: "491c9ee2a9dbb6ad4e71f9707d848b55"
server_key_iv: "652ba7870d6291119c95cbd641f3fdf9"
server_key_version: "encryptionkey@0"
I20260929 02:57:05.286861 28318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:05.301493 28318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:05.302371 28318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:05.302644 28318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:05.303650 28318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:05.305212 28318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:05.305310 28318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:05.305431 28318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:05.305500 28318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:05.336325 28318 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.1:45185
I20260929 02:57:05.336371 28444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.1:45185 every 8 connection(s)
I20260929 02:57:05.337611 28318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/data/info.pb
I20260929 02:57:05.344115 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28318
I20260929 02:57:05.344252 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal/instance
I20260929 02:57:05.344668 8420 external_mini_cluster.cc:1428] Setting key 6336b4c883f19c87645bd35a57aea17f
I20260929 02:57:05.347229 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.2:0
--local_ip_for_outbound_sockets=127.8.57.2
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34165
--builtin_ntp_servers=127.8.57.20:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:57:05.349900 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:05.339989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.1:56535 (local address 127.8.57.62:34165)
0929 02:57:05.340434 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:05.340440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:05.341589 (+ 1149us) server_negotiation.cc:407] Connection header received
0929 02:57:05.343310 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:05.343314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:05.343379 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:05.343469 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:05.344607 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.345480 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.347131 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.347619 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.347723 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:05.349071 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:05.349076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:05.349080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:05.349096 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:05.349148 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:05.349165 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:05.349245 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:05.349469 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 02:57:05.349573 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":208,"threads_started":1}
I20260929 02:57:05.350870 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:05.340595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34165 (local address 127.8.57.1:56535)
0929 02:57:05.341448 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:05.341487 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:05.343071 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:05.343651 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:05.343659 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:05.343744 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:05.344423 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:05.344445 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.345628 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.345633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:05.346990 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:05.347003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.347834 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.347841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:05.348099 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:05.348757 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:57:05.348760 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:05.348799 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:57:05.348872 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:05.349245 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:05.349257 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:05.349265 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:57:05.349419 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 02:57:05.349797 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":255,"threads_started":1}
I20260929 02:57:05.352368 28445 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34165
I20260929 02:57:05.352626 28445 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:05.353410 28445 heartbeater.cc:511] Master 127.8.57.62:34165 requested a full tablet report, sending...
I20260929 02:57:05.355322 28259 ts_manager.cc:194] Registered new tserver with Master: 5665a13f9de64ec5b4bda85d173ccdda (127.8.57.1:45185)
I20260929 02:57:05.356935 28259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.1:56535
W20260929 02:57:05.661738 28449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:05.661998 28449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:05.662122 28449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:05.669327 28449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:05.669422 28449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:05.669555 28449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.2
I20260929 02:57:05.678536 28449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.57.2
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.2
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:05.679795 28449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:05.681543 28449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:05.693693 28454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:05.693738 28455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:05.693773 28457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:05.694659 28449 server_base.cc:1034] running on GCE node
I20260929 02:57:05.695215 28449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:05.696112 28449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:05.697301 28449 hybrid_clock.cc:648] HybridClock initialized: now 1790650625697253 us; error 44 us; skew 500 ppm
I20260929 02:57:05.697587 28449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:57:05.699713 28449 webserver.cc:460] Webserver started at http://127.8.57.2:45683/ using document root <none> and password file <none>
I20260929 02:57:05.700395 28449 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:05.700495 28449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:05.700793 28449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:05.703433 28449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data/instance:
uuid: "c59a3ecb6e3242319955d251e8b21a9c"
format_stamp: "Formatted at 2026-09-29 02:57:05 on dist-test-slave-1dkt"
server_key: "44ef8e0041a8ca5b54f26c112714a20f"
server_key_iv: "65b1c9e62df3a959a9998501b68b1648"
server_key_version: "encryptionkey@0"
I20260929 02:57:05.704193 28449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance:
uuid: "c59a3ecb6e3242319955d251e8b21a9c"
format_stamp: "Formatted at 2026-09-29 02:57:05 on dist-test-slave-1dkt"
server_key: "44ef8e0041a8ca5b54f26c112714a20f"
server_key_iv: "65b1c9e62df3a959a9998501b68b1648"
server_key_version: "encryptionkey@0"
I20260929 02:57:05.708783 28449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:57:05.712036 28463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:05.713286 28449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:57:05.713429 28449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal
uuid: "c59a3ecb6e3242319955d251e8b21a9c"
format_stamp: "Formatted at 2026-09-29 02:57:05 on dist-test-slave-1dkt"
server_key: "44ef8e0041a8ca5b54f26c112714a20f"
server_key_iv: "65b1c9e62df3a959a9998501b68b1648"
server_key_version: "encryptionkey@0"
I20260929 02:57:05.713616 28449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:05.735217 28449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:05.736075 28449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:05.736338 28449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:05.737303 28449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:05.738790 28449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:05.738883 28449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:05.738989 28449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:05.739055 28449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:05.769215 28449 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.2:45189
I20260929 02:57:05.769285 28575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.2:45189 every 8 connection(s)
I20260929 02:57:05.770483 28449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/data/info.pb
I20260929 02:57:05.780716 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28449
I20260929 02:57:05.780889 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal/instance
I20260929 02:57:05.781445 8420 external_mini_cluster.cc:1428] Setting key 6ec5a42a6b82e0717ed8463b0d3e8825
I20260929 02:57:05.782346 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:05.772901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.2:54137 (local address 127.8.57.62:34165)
0929 02:57:05.773116 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:05.773122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:05.774500 (+ 1378us) server_negotiation.cc:407] Connection header received
0929 02:57:05.776260 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:05.776265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:05.776334 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:05.776437 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:05.777415 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.778097 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.779566 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.780118 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.780235 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:05.781585 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:05.781592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:05.781597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:05.781616 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:05.781648 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:05.781661 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:05.781728 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:05.781954 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 02:57:05.782061 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:57:05.783456 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:05.773452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34165 (local address 127.8.57.2:54137)
0929 02:57:05.774353 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:05.774392 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:05.776019 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:05.776596 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:05.776604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:05.776694 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:05.777257 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:05.777273 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.778210 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.778213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:05.779447 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:05.779459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:05.780357 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:05.780363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:05.780566 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:05.781232 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:57:05.781235 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:05.781285 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:57:05.781383 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:05.781750 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:05.781760 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:05.781768 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:57:05.781946 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 02:57:05.782266 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":261,"threads_started":1}
I20260929 02:57:05.784262 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.57.3:0
--local_ip_for_outbound_sockets=127.8.57.3
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34165
--builtin_ntp_servers=127.8.57.20:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:57:05.784781 28576 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34165
I20260929 02:57:05.785089 28576 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:05.785915 28576 heartbeater.cc:511] Master 127.8.57.62:34165 requested a full tablet report, sending...
I20260929 02:57:05.787252 28259 ts_manager.cc:194] Registered new tserver with Master: c59a3ecb6e3242319955d251e8b21a9c (127.8.57.2:45189)
I20260929 02:57:05.788100 28259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.2:54137
W20260929 02:57:06.102437 28579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:06.102717 28579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:06.102802 28579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:06.110013 28579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:06.110111 28579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:06.110214 28579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.57.3
I20260929 02:57:06.119261 28579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.57.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.57.3
--webserver_port=0
--tserver_master_addrs=127.8.57.62:34165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.57.3
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:06.120465 28579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:06.122130 28579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:06.135066 28584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:06.135087 28587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:06.135030 28585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:06.135782 28579 server_base.cc:1034] running on GCE node
I20260929 02:57:06.136457 28579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:06.137394 28579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:06.138592 28579 hybrid_clock.cc:648] HybridClock initialized: now 1790650626138539 us; error 48 us; skew 500 ppm
I20260929 02:57:06.138891 28579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:57:06.141105 28579 webserver.cc:460] Webserver started at http://127.8.57.3:43919/ using document root <none> and password file <none>
I20260929 02:57:06.141755 28579 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:06.141855 28579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:06.142156 28579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:06.144925 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data/instance:
uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee"
format_stamp: "Formatted at 2026-09-29 02:57:06 on dist-test-slave-1dkt"
server_key: "e40b40adbfa4377430cb931b3e995555"
server_key_iv: "b52422e462572bedbfb43d90985a4e71"
server_key_version: "encryptionkey@0"
I20260929 02:57:06.145663 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance:
uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee"
format_stamp: "Formatted at 2026-09-29 02:57:06 on dist-test-slave-1dkt"
server_key: "e40b40adbfa4377430cb931b3e995555"
server_key_iv: "b52422e462572bedbfb43d90985a4e71"
server_key_version: "encryptionkey@0"
I20260929 02:57:06.150494 28579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:57:06.153868 28593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:06.155225 28579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:06.155346 28579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal
uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee"
format_stamp: "Formatted at 2026-09-29 02:57:06 on dist-test-slave-1dkt"
server_key: "e40b40adbfa4377430cb931b3e995555"
server_key_iv: "b52422e462572bedbfb43d90985a4e71"
server_key_version: "encryptionkey@0"
I20260929 02:57:06.155526 28579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:06.183511 28579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:06.184350 28579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:06.184590 28579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:06.185487 28579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:06.186857 28579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:06.186942 28579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:06.187023 28579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:06.187088 28579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:06.214632 28579 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.3:42255
I20260929 02:57:06.214679 28705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.3:42255 every 8 connection(s)
I20260929 02:57:06.215958 28579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/data/info.pb
I20260929 02:57:06.217886 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28579
I20260929 02:57:06.218045 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal/instance
I20260929 02:57:06.218513 8420 external_mini_cluster.cc:1428] Setting key ce216a87958e1d5e1ae1b93114b37f7f
I20260929 02:57:06.228123 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.218595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:53617 (local address 127.8.57.62:34165)
0929 02:57:06.218795 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.218801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.220258 (+ 1457us) server_negotiation.cc:407] Connection header received
0929 02:57:06.222155 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.222159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.222222 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.222302 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:06.223367 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.224157 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.225643 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.226054 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.226147 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.227318 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:06.227323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:06.227327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:06.227344 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:06.227380 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:06.227394 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:06.227470 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.227708 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.227784 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:57:06.228931 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.219090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:34165 (local address 127.8.57.3:53617)
0929 02:57:06.220114 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:06.220156 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:06.221930 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:06.222560 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:06.222568 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:06.222654 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:06.223207 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.223229 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.224285 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.224289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.225518 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.225530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.226247 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.226252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.226438 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.227017 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:57:06.227019 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:06.227061 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:57:06.227130 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:06.227485 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:06.227491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:06.227497 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:57:06.227636 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 02:57:06.227944 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":301,"threads_started":1}
I20260929 02:57:06.230068 28706 heartbeater.cc:347] Connected to a master server at 127.8.57.62:34165
I20260929 02:57:06.230312 28706 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:06.230963 28706 heartbeater.cc:511] Master 127.8.57.62:34165 requested a full tablet report, sending...
I20260929 02:57:06.232365 28258 ts_manager.cc:194] Registered new tserver with Master: d1d8c98e77ff45c8b2f043d4cb7fdbee (127.8.57.3:42255)
I20260929 02:57:06.233402 8420 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:57:06.233420 28258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.57.3:53617
I20260929 02:57:06.242568 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.237314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.8.57.62:34165)
0929 02:57:06.237447 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.237452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.237899 (+ 447us) server_negotiation.cc:407] Connection header received
0929 02:57:06.238027 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.238030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.238101 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.238176 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:06.238579 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.239224 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.240565 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.240963 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.241047 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.242072 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:06.242077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:06.242080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:06.242096 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:06.242126 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:06.242137 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:06.242195 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.242279 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.242350 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:57:06.246064 28258 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:57:06.252923 28258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:57:06.276415 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.265064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.8.57.1:45185)
0929 02:57:06.265832 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.265839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.266980 (+ 1141us) server_negotiation.cc:407] Connection header received
0929 02:57:06.267087 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.267094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.267312 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.267517 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:06.268560 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.270220 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.273285 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.274738 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.274873 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.275123 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.275891 (+ 768us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.276074 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":496,"threads_started":1}
I20260929 02:57:06.276504 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.264552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:45185 (local address 127.0.0.1:51722)
0929 02:57:06.266700 (+ 2148us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:06.266721 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:06.266836 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:06.267771 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:06.267779 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:06.267824 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:06.268317 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.268334 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.270384 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.270390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.273115 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.273126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.275265 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.275275 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.275579 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.275666 (+ 87us) client_negotiation.cc:769] Sending connection context
0929 02:57:06.275801 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 02:57:06.276046 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2027,"mutex_wait_us":114,"spinlock_wait_cycles":4352,"thread_start_us":2532,"threads_started":1}
I20260929 02:57:06.276752 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.264313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:45189 (local address 127.0.0.1:55468)
0929 02:57:06.266064 (+ 1751us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:06.266105 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:06.266368 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:06.267783 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:06.267788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:06.267838 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:06.268482 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.268495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.270554 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.270563 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.273309 (+ 2746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.273324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.275295 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.275302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.275560 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.275666 (+ 106us) client_negotiation.cc:769] Sending connection context
0929 02:57:06.275801 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 02:57:06.276046 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1523,"mutex_wait_us":61,"spinlock_wait_cycles":3840,"thread_start_us":1396,"threads_started":1}
I20260929 02:57:06.277472 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.264633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.8.57.2:45189)
0929 02:57:06.265580 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.265590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.266206 (+ 616us) server_negotiation.cc:407] Connection header received
0929 02:57:06.266635 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.266645 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.266917 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.267081 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:06.268673 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.270352 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.273496 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.274796 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.274904 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.275155 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.276940 (+ 1785us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.277122 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":645,"threads_started":1}
I20260929 02:57:06.278460 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.266526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.3:42255 (local address 127.0.0.1:40432)
0929 02:57:06.271340 (+ 4814us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:06.271354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:06.271470 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:06.272206 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:06.272211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:06.272236 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:06.272663 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.272678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.274227 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.274232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.276016 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.276029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.277963 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.277968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.278092 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.278115 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:57:06.278181 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 02:57:06.278262 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4716,"thread_start_us":4568,"threads_started":1}
I20260929 02:57:06.278786 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.266807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.8.57.3:42255)
0929 02:57:06.271054 (+ 4247us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.271062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.271448 (+ 386us) server_negotiation.cc:407] Connection header received
0929 02:57:06.271657 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.271665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.271904 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.272197 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:06.272865 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.274103 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.276229 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.277554 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.277676 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.277870 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.278275 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.278451 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4097,"thread_start_us":3963,"threads_started":1}
I20260929 02:57:06.280359 28380 tablet_service.cc:1467] Processing CreateTablet for tablet e9cd1982160241b4a7040e04dc77b8eb (DEFAULT_TABLE table=test-table [id=1a8fe187e12f4a6e9e6c95247dae02ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:06.281507 28380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9cd1982160241b4a7040e04dc77b8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:06.281677 28641 tablet_service.cc:1467] Processing CreateTablet for tablet e9cd1982160241b4a7040e04dc77b8eb (DEFAULT_TABLE table=test-table [id=1a8fe187e12f4a6e9e6c95247dae02ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:06.282724 28641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9cd1982160241b4a7040e04dc77b8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:06.286995 28511 tablet_service.cc:1467] Processing CreateTablet for tablet e9cd1982160241b4a7040e04dc77b8eb (DEFAULT_TABLE table=test-table [id=1a8fe187e12f4a6e9e6c95247dae02ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:06.288326 28511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9cd1982160241b4a7040e04dc77b8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:06.293447 28724 tablet_bootstrap.cc:492] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Bootstrap starting.
I20260929 02:57:06.296396 28725 tablet_bootstrap.cc:492] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Bootstrap starting.
I20260929 02:57:06.297101 28724 tablet_bootstrap.cc:654] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:06.298324 28724 log.cc:824] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:06.300402 28725 tablet_bootstrap.cc:654] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:06.301399 28724 tablet_bootstrap.cc:492] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: No bootstrap required, opened a new log
I20260929 02:57:06.301533 28725 log.cc:824] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:06.301657 28724 ts_tablet_manager.cc:1397] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:57:06.301774 28727 tablet_bootstrap.cc:492] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Bootstrap starting.
I20260929 02:57:06.304661 28727 tablet_bootstrap.cc:654] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:06.305795 28727 log.cc:824] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:06.306290 28725 tablet_bootstrap.cc:492] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: No bootstrap required, opened a new log
I20260929 02:57:06.306546 28725 ts_tablet_manager.cc:1397] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260929 02:57:06.309715 28727 tablet_bootstrap.cc:492] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: No bootstrap required, opened a new log
I20260929 02:57:06.309937 28727 ts_tablet_manager.cc:1397] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260929 02:57:06.311303 28724 raft_consensus.cc:348] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.311709 28724 raft_consensus.cc:374] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:06.311806 28724 raft_consensus.cc:729] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d8c98e77ff45c8b2f043d4cb7fdbee, State: Initialized, Role: FOLLOWER
I20260929 02:57:06.312410 28724 consensus_queue.cc:260] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.313445 28706 heartbeater.cc:503] Master 127.8.57.62:34165 was elected leader, sending a full tablet report...
I20260929 02:57:06.314028 28724 ts_tablet_manager.cc:1428] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260929 02:57:06.316702 28725 raft_consensus.cc:348] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.317248 28725 raft_consensus.cc:374] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:06.317356 28725 raft_consensus.cc:729] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5665a13f9de64ec5b4bda85d173ccdda, State: Initialized, Role: FOLLOWER
I20260929 02:57:06.317898 28725 consensus_queue.cc:260] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.317888 28727 raft_consensus.cc:348] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.318310 28727 raft_consensus.cc:374] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:06.318406 28727 raft_consensus.cc:729] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59a3ecb6e3242319955d251e8b21a9c, State: Initialized, Role: FOLLOWER
I20260929 02:57:06.318876 28727 consensus_queue.cc:260] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.319785 28576 heartbeater.cc:503] Master 127.8.57.62:34165 was elected leader, sending a full tablet report...
I20260929 02:57:06.320428 28445 heartbeater.cc:503] Master 127.8.57.62:34165 was elected leader, sending a full tablet report...
I20260929 02:57:06.320822 28725 ts_tablet_manager.cc:1428] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260929 02:57:06.321498 28727 ts_tablet_manager.cc:1428] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
W20260929 02:57:06.339619 28446 tablet.cc:2369] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:57:06.467700 28707 tablet.cc:2369] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:57:06.522389 28577 tablet.cc:2369] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:57:06.646360 28730 raft_consensus.cc:482] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:57:06.646646 28730 raft_consensus.cc:504] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.648339 28730 leader_election.cc:288] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c59a3ecb6e3242319955d251e8b21a9c (127.8.57.2:45189), 5665a13f9de64ec5b4bda85d173ccdda (127.8.57.1:45185)
I20260929 02:57:06.654273 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.648834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:46319 (local address 127.8.57.2:45189)
0929 02:57:06.649028 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.649033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.649190 (+ 157us) server_negotiation.cc:407] Connection header received
0929 02:57:06.649440 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.649445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.649530 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.649645 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:06.650091 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.650888 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.652602 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.653454 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.653567 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.653663 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.653898 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.654024 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:57:06.654300 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.648768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.2:45189 (local address 127.8.57.3:46319)
0929 02:57:06.649080 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:06.649101 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:06.649285 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:06.649716 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:06.649719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:06.649744 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:06.649940 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.649949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.651015 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.651019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.652447 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.652459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.653597 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.653604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.653755 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.653779 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:57:06.653857 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:57:06.654009 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":18,"spinlock_wait_cycles":17152}
I20260929 02:57:06.654708 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.648846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.57.3:56003 (local address 127.8.57.1:45185)
0929 02:57:06.649025 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.649031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.649684 (+ 653us) server_negotiation.cc:407] Connection header received
0929 02:57:06.649919 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.649924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.649999 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.650068 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:06.650557 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.651258 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.652968 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.653841 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.653950 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.654044 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.654344 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.654466 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:57:06.654820 28733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.648764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.1:45185 (local address 127.8.57.3:56003)
0929 02:57:06.649588 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:06.649606 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:06.649784 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:06.650103 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:06.650108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:06.650129 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:06.650417 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.650426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.651404 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.651411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.652805 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:06.652816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.654018 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.654024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:06.654156 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.654178 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:57:06.654317 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 02:57:06.654440 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":34,"thread_start_us":428,"threads_started":1}
I20260929 02:57:06.655570 28531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9cd1982160241b4a7040e04dc77b8eb" candidate_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59a3ecb6e3242319955d251e8b21a9c" is_pre_election: true
I20260929 02:57:06.655839 28400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9cd1982160241b4a7040e04dc77b8eb" candidate_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5665a13f9de64ec5b4bda85d173ccdda" is_pre_election: true
I20260929 02:57:06.656271 28531 raft_consensus.cc:2454] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d8c98e77ff45c8b2f043d4cb7fdbee in term 0.
I20260929 02:57:06.656395 28400 raft_consensus.cc:2454] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d8c98e77ff45c8b2f043d4cb7fdbee in term 0.
I20260929 02:57:06.657040 28596 leader_election.cc:302] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5665a13f9de64ec5b4bda85d173ccdda, c59a3ecb6e3242319955d251e8b21a9c, d1d8c98e77ff45c8b2f043d4cb7fdbee; no voters:
I20260929 02:57:06.657455 28730 raft_consensus.cc:2781] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:57:06.657582 28730 raft_consensus.cc:482] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:57:06.657675 28730 raft_consensus.cc:3037] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:06.659848 28730 raft_consensus.cc:504] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.660436 28730 leader_election.cc:288] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [CANDIDATE]: Term 1 election: Requested vote from peers c59a3ecb6e3242319955d251e8b21a9c (127.8.57.2:45189), 5665a13f9de64ec5b4bda85d173ccdda (127.8.57.1:45185)
I20260929 02:57:06.660866 28531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9cd1982160241b4a7040e04dc77b8eb" candidate_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59a3ecb6e3242319955d251e8b21a9c"
I20260929 02:57:06.660925 28400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9cd1982160241b4a7040e04dc77b8eb" candidate_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5665a13f9de64ec5b4bda85d173ccdda"
I20260929 02:57:06.661053 28531 raft_consensus.cc:3037] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:06.661109 28400 raft_consensus.cc:3037] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:06.663302 28531 raft_consensus.cc:2454] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d8c98e77ff45c8b2f043d4cb7fdbee in term 1.
I20260929 02:57:06.663342 28400 raft_consensus.cc:2454] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d8c98e77ff45c8b2f043d4cb7fdbee in term 1.
I20260929 02:57:06.663681 28597 leader_election.cc:302] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c59a3ecb6e3242319955d251e8b21a9c, d1d8c98e77ff45c8b2f043d4cb7fdbee; no voters:
I20260929 02:57:06.663961 28730 raft_consensus.cc:2781] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:06.664348 28730 raft_consensus.cc:686] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 1 LEADER]: Becoming Leader. State: Replica: d1d8c98e77ff45c8b2f043d4cb7fdbee, State: Running, Role: LEADER
I20260929 02:57:06.664780 28730 consensus_queue.cc:237] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } }
I20260929 02:57:06.668875 28258 catalog_manager.cc:5495] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee reported cstate change: term changed from 0 to 1, leader changed from <none> to d1d8c98e77ff45c8b2f043d4cb7fdbee (127.8.57.3). New cstate: current_term: 1 leader_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1d8c98e77ff45c8b2f043d4cb7fdbee" member_type: VOTER last_known_addr { host: "127.8.57.3" port: 42255 } health_report { overall_health: HEALTHY } } }
I20260929 02:57:06.730253 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.724944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.8.57.3:42255)
0929 02:57:06.725141 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.725147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.725256 (+ 109us) server_negotiation.cc:407] Connection header received
0929 02:57:06.725487 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.725492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.725581 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.725712 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:06.726075 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.726795 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.728248 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.728655 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.728759 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.729230 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:06.729708 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:06.729849 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.729933 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.730013 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 02:57:06.738644 28400 raft_consensus.cc:1261] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 1 FOLLOWER]: Refusing update from remote peer d1d8c98e77ff45c8b2f043d4cb7fdbee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:06.738644 28531 raft_consensus.cc:1261] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 1 FOLLOWER]: Refusing update from remote peer d1d8c98e77ff45c8b2f043d4cb7fdbee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:06.739534 28734 consensus_queue.cc:1035] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59a3ecb6e3242319955d251e8b21a9c" member_type: VOTER last_known_addr { host: "127.8.57.2" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:06.739895 28730 consensus_queue.cc:1035] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5665a13f9de64ec5b4bda85d173ccdda" member_type: VOTER last_known_addr { host: "127.8.57.1" port: 45185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:06.751982 28738 mvcc.cc:205] Tried to move back new op lower bound from 7334504967111270400 to 7334504966822473728. Current Snapshot: MvccSnapshot[applied={T|T < 7334504967111270400}]
I20260929 02:57:06.765597 28259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:57:06.765883 28259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:57:06.769466 28259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f16bcb69bb54ba6828ce5b47fdb4177: Sending DeleteTablet for 3 replicas of tablet e9cd1982160241b4a7040e04dc77b8eb
I20260929 02:57:06.770936 28641 tablet_service.cc:1514] Processing DeleteTablet for tablet e9cd1982160241b4a7040e04dc77b8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:57:06 UTC) from {username='slave'} at 127.0.0.1:40432
I20260929 02:57:06.771032 28511 tablet_service.cc:1514] Processing DeleteTablet for tablet e9cd1982160241b4a7040e04dc77b8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:57:06 UTC) from {username='slave'} at 127.0.0.1:55468
I20260929 02:57:06.771730 28380 tablet_service.cc:1514] Processing DeleteTablet for tablet e9cd1982160241b4a7040e04dc77b8eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:57:06 UTC) from {username='slave'} at 127.0.0.1:51722
I20260929 02:57:06.772655 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28318
I20260929 02:57:06.795822 28749 tablet_replica.cc:330] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: stopping tablet replica
I20260929 02:57:06.796478 28749 raft_consensus.cc:2229] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:57:06.797283 28748 tablet_replica.cc:330] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: stopping tablet replica
I20260929 02:57:06.797899 28749 raft_consensus.cc:2258] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:57:06.798393 28748 raft_consensus.cc:2229] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:57:06.799213 28748 raft_consensus.cc:2258] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:57:06.801338 28749 ts_tablet_manager.cc:1905] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:57:06.801520 28748 ts_tablet_manager.cc:1905] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:57:06.802184 28747 tablet_replica.cc:330] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: stopping tablet replica
I20260929 02:57:06.802657 28747 raft_consensus.cc:2229] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:57:06.803328 28747 raft_consensus.cc:2258] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:57:06.805200 28747 ts_tablet_manager.cc:1905] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:57:06.808681 28748 ts_tablet_manager.cc:1918] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:57:06.809029 28749 ts_tablet_manager.cc:1918] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:57:06.809252 28749 log.cc:1196] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-0/wal/wals/e9cd1982160241b4a7040e04dc77b8eb
I20260929 02:57:06.809839 28749 ts_tablet_manager.cc:1939] T e9cd1982160241b4a7040e04dc77b8eb P 5665a13f9de64ec5b4bda85d173ccdda: Deleting consensus metadata
I20260929 02:57:06.810855 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.774287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.8.57.1:45185)
0929 02:57:06.774446 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.774452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.796392 (+ 21940us) server_negotiation.cc:407] Connection header received
0929 02:57:06.796695 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.796701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.796826 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.796904 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:06.797586 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.798923 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.800775 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.801344 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.801438 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.809233 (+ 7795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:06.809247 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:06.809264 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:06.810009 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:06.810066 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:06.810090 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:06.810179 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.810423 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.810544 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:57:06.811760 28748 log.cc:1196] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-1/wal/wals/e9cd1982160241b4a7040e04dc77b8eb
I20260929 02:57:06.812266 28747 ts_tablet_manager.cc:1918] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:57:06.812300 28748 ts_tablet_manager.cc:1939] T e9cd1982160241b4a7040e04dc77b8eb P c59a3ecb6e3242319955d251e8b21a9c: Deleting consensus metadata
I20260929 02:57:06.812493 28747 log.cc:1196] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Deleting WAL directory at /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650321430198-8420-0/minicluster-data/ts-2/wal/wals/e9cd1982160241b4a7040e04dc77b8eb
I20260929 02:57:06.813130 28747 ts_tablet_manager.cc:1939] T e9cd1982160241b4a7040e04dc77b8eb P d1d8c98e77ff45c8b2f043d4cb7fdbee: Deleting consensus metadata
I20260929 02:57:06.813866 28246 catalog_manager.cc:4841] TS c59a3ecb6e3242319955d251e8b21a9c (127.8.57.2:45189): tablet e9cd1982160241b4a7040e04dc77b8eb (table test-table [id=1a8fe187e12f4a6e9e6c95247dae02ed]) successfully deleted
I20260929 02:57:06.814738 28244 catalog_manager.cc:4841] TS d1d8c98e77ff45c8b2f043d4cb7fdbee (127.8.57.3:42255): tablet e9cd1982160241b4a7040e04dc77b8eb (table test-table [id=1a8fe187e12f4a6e9e6c95247dae02ed]) successfully deleted
I20260929 02:57:06.814949 28440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:06.925742 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28318
I20260929 02:57:06.925830 28245 catalog_manager.cc:4841] TS 5665a13f9de64ec5b4bda85d173ccdda (127.8.57.1:45185): tablet e9cd1982160241b4a7040e04dc77b8eb (table test-table [id=1a8fe187e12f4a6e9e6c95247dae02ed]) successfully deleted
I20260929 02:57:06.946292 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28449
I20260929 02:57:06.953428 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:06.947162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.8.57.2:45189)
0929 02:57:06.947334 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:06.947339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:06.947359 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:57:06.947510 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:06.947514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:06.947609 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:06.947694 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:06.948177 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.948860 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.950148 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:06.950616 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:06.950725 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:06.951828 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:06.951837 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:06.951852 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:06.952693 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:06.952740 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:06.952761 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:06.952869 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:06.952994 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 02:57:06.953109 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:57:06.954051 28571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:07.065716 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28449
I20260929 02:57:07.080528 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28579
I20260929 02:57:07.088204 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:07.081267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40458 (local address 127.8.57.3:42255)
0929 02:57:07.081426 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:07.081430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:07.081493 (+ 63us) server_negotiation.cc:407] Connection header received
0929 02:57:07.081687 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:07.081691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:07.081771 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:07.081859 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:07.082341 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:07.083171 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:07.084967 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:07.085433 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:07.085525 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:07.086685 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:07.086693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:07.086707 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:07.087336 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:07.087388 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:07.087409 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:07.087497 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:07.087692 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 02:57:07.087779 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:57:07.089005 28701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:07.211133 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28579
I20260929 02:57:07.226181 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28228
I20260929 02:57:07.227236 28289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:07.334901 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28228
2026-09-29T02:57:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3015 ms)
[----------] 6 tests from EncryptionPolicyTest (20510 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:07 dist-test-slave-1dkt krb5kdc[28757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:07 dist-test-slave-1dkt krb5kdc[28757](info): set up 2 sockets
Sep 29 02:57:07 dist-test-slave-1dkt krb5kdc[28757](info): commencing operation
krb5kdc: starting...
W20260929 02:57:08.356262 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:08 dist-test-slave-1dkt krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:08.579633 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:08.892946 28770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:08.893232 28770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:08.893355 28770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:08.900493 28770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:08.900602 28770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:08.900676 28770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:08.900756 28770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:08.900812 28770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:08.910439 28770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:08.911667 28770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:08.912936 28770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:08.923699 28776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:08.923825 28778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:08.923852 28775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:08.925060 28770 server_base.cc:1034] running on GCE node
I20260929 02:57:08.925750 28770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:08.927043 28770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:08.928242 28770 hybrid_clock.cc:648] HybridClock initialized: now 1790650628928192 us; error 45 us; skew 500 ppm
I20260929 02:57:08.928581 28770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:08 dist-test-slave-1dkt krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:08.931399 28770 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:08.932783 28770 webserver.cc:460] Webserver started at http://127.0.0.1:36227/ using document root <none> and password file <none>
I20260929 02:57:08.933461 28770 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:08.933574 28770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:08.933840 28770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:08.936745 28770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "6f686fa6a146440bbaaf4fb03ba25fd0"
format_stamp: "Formatted at 2026-09-29 02:57:08 on dist-test-slave-1dkt"
server_key: "9b68017832755f1e3abfa39496206a8a"
server_key_iv: "f88ad9b46f3b2973b9e325323877b02f"
server_key_version: "encryptionkey@0"
I20260929 02:57:08.937481 28770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "6f686fa6a146440bbaaf4fb03ba25fd0"
format_stamp: "Formatted at 2026-09-29 02:57:08 on dist-test-slave-1dkt"
server_key: "9b68017832755f1e3abfa39496206a8a"
server_key_iv: "f88ad9b46f3b2973b9e325323877b02f"
server_key_version: "encryptionkey@0"
I20260929 02:57:08.942313 28770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:57:08.945659 28785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:08.946902 28770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:57:08.947064 28770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "6f686fa6a146440bbaaf4fb03ba25fd0"
format_stamp: "Formatted at 2026-09-29 02:57:08 on dist-test-slave-1dkt"
server_key: "9b68017832755f1e3abfa39496206a8a"
server_key_iv: "f88ad9b46f3b2973b9e325323877b02f"
server_key_version: "encryptionkey@0"
I20260929 02:57:08.947239 28770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:08.975238 28770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:08.978914 28770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:08.979177 28770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:08.995807 28770 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37019
I20260929 02:57:08.995826 28836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37019 every 8 connection(s)
I20260929 02:57:08.997099 28770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:09.000322 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28770
I20260929 02:57:09.000550 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:09.000849 28837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:09.001322 8420 external_mini_cluster.cc:1428] Setting key b1422b52185f7534109589bebc0a40a0
I20260929 02:57:09.014685 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0: Bootstrap starting.
Sep 29 02:57:09 dist-test-slave-1dkt krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:57:09.018466 28837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:09.019764 28837 log.cc:824] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:09.023295 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0: No bootstrap required, opened a new log
I20260929 02:57:09.025019 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:09.003180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39753 (local address 127.0.0.1:37019)
0929 02:57:09.003940 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:09.003954 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:09.004005 (+ 51us) server_negotiation.cc:407] Connection header received
0929 02:57:09.005696 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:09.005723 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:09.006201 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:09.006619 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:09.007834 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:09.009806 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:09.011103 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:09.012188 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:09.012361 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:09.016313 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:09.016324 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:09.016342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:09.016398 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:09.020141 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:09.021193 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:09.021200 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:09.021207 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:09.021434 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:09.021878 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:09.021883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:09.021886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:09.022592 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:09.022891 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:09.023343 (+ 452us) server_negotiation.cc:299] Negotiation successful
0929 02:57:09.023692 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":302,"threads_started":1}
I20260929 02:57:09.029042 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:09.031910 28837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260929 02:57:09.032311 28837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:09.032413 28837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f686fa6a146440bbaaf4fb03ba25fd0, State: Initialized, Role: FOLLOWER
I20260929 02:57:09.032940 28837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260929 02:57:09.033157 28837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:09.033279 28837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:09.033448 28837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:09.035748 28837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260929 02:57:09.036386 28837 leader_election.cc:302] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f686fa6a146440bbaaf4fb03ba25fd0; no voters:
I20260929 02:57:09.037010 28837 leader_election.cc:288] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:09.037143 28847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:09.038911 28837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:09.039179 28847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6f686fa6a146440bbaaf4fb03ba25fd0, State: Running, Role: LEADER
I20260929 02:57:09.039819 28847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260929 02:57:09.043718 28849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f686fa6a146440bbaaf4fb03ba25fd0. Latest consensus state: current_term: 1 leader_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } } }
I20260929 02:57:09.044822 28849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:09.044272 28848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f686fa6a146440bbaaf4fb03ba25fd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } } }
I20260929 02:57:09.045364 28848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:09.049738 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:09.033591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51453 (local address 127.0.0.1:37019)
0929 02:57:09.033764 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:09.033770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:09.034218 (+ 448us) server_negotiation.cc:407] Connection header received
0929 02:57:09.034345 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:09.034350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:09.034428 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:09.034544 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:09.035839 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:09.036988 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:09.038939 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:09.039613 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:09.039791 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:09.042543 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:09.042550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:09.042554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:09.042594 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:09.046221 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:09.047302 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:09.047308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:09.047312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:09.047466 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:09.048068 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:09.048074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:09.048077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:09.048520 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:09.048774 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:09.049212 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 02:57:09.049376 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":55}
I20260929 02:57:09.051595 28854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:09.057058 28854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:09.065793 28854 catalog_manager.cc:1347] Generated new cluster ID: 313c976ebadd4850813035527e0854d7
I20260929 02:57:09.065894 28854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:09.087046 28854 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:09.088920 28854 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:09.097000 28854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f686fa6a146440bbaaf4fb03ba25fd0: Generated new TSK 0
I20260929 02:57:09.097997 28854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:09.112607 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28770
I20260929 02:57:09.113907 28832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:09.209394 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28770
W20260929 02:57:09.223040 28755 connection.cc:535] client connection to 127.0.0.1:37019 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37019): Connection reset by peer (error 104)
2026-09-29T02:57:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1870 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:09 dist-test-slave-1dkt krb5kdc[28869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:09 dist-test-slave-1dkt krb5kdc[28869](info): set up 2 sockets
Sep 29 02:57:09 dist-test-slave-1dkt krb5kdc[28869](info): commencing operation
krb5kdc: starting...
W20260929 02:57:10.228114 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:10 dist-test-slave-1dkt krb5kdc[28869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:10.453400 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:10.762555 28882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:10.762850 28882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:10.762991 28882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:10.770193 28882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:10.770321 28882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:10.770401 28882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:10.770478 28882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:10.770540 28882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:10.780246 28882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:10.781471 28882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:10.782701 28882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:10.793761 28887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:10.793803 28890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:10.793735 28888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:10.794734 28882 server_base.cc:1034] running on GCE node
I20260929 02:57:10.795506 28882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:10.796940 28882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:10.798146 28882 hybrid_clock.cc:648] HybridClock initialized: now 1790650630798106 us; error 32 us; skew 500 ppm
I20260929 02:57:10.798461 28882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:10 dist-test-slave-1dkt krb5kdc[28869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:10.801463 28882 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:10.802850 28882 webserver.cc:460] Webserver started at http://127.0.0.1:39407/ using document root <none> and password file <none>
I20260929 02:57:10.803578 28882 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:10.803704 28882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:10.804031 28882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:10.806906 28882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "c95f13efd99246769623f4f32dea0987"
format_stamp: "Formatted at 2026-09-29 02:57:10 on dist-test-slave-1dkt"
server_key: "3dedc4f95a2368df14dc962e2343ff75"
server_key_iv: "a59fb2ba550dbeffc2a411ead07f6415"
server_key_version: "encryptionkey@0"
I20260929 02:57:10.807644 28882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "c95f13efd99246769623f4f32dea0987"
format_stamp: "Formatted at 2026-09-29 02:57:10 on dist-test-slave-1dkt"
server_key: "3dedc4f95a2368df14dc962e2343ff75"
server_key_iv: "a59fb2ba550dbeffc2a411ead07f6415"
server_key_version: "encryptionkey@0"
I20260929 02:57:10.812484 28882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:57:10.815769 28897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:10.817067 28882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:57:10.817233 28882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "c95f13efd99246769623f4f32dea0987"
format_stamp: "Formatted at 2026-09-29 02:57:10 on dist-test-slave-1dkt"
server_key: "3dedc4f95a2368df14dc962e2343ff75"
server_key_iv: "a59fb2ba550dbeffc2a411ead07f6415"
server_key_version: "encryptionkey@0"
I20260929 02:57:10.817404 28882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:10.848588 28882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:10.852887 28882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:10.853155 28882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:10.870446 28882 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41371
I20260929 02:57:10.870491 28948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41371 every 8 connection(s)
I20260929 02:57:10.871726 28882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:10.873865 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28882
I20260929 02:57:10.874037 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:10.874464 8420 external_mini_cluster.cc:1428] Setting key 17c7eed3700942f53ef6bc040969d55f
I20260929 02:57:10.875485 28949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:10.888442 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:10.876153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42425 (local address 127.0.0.1:41371)
0929 02:57:10.876916 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:10.876930 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:10.876980 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:57:10.878715 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:10.878742 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:10.879231 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:10.879657 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:10.880151 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:10.882094 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:10.883501 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:10.884524 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:10.884699 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:10.885801 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:10.885812 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:10.885829 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:10.886528 (+ 699us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:10.886620 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:10.886656 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:10.886817 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:10.887025 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 02:57:10.887196 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":309,"threads_started":1}
I20260929 02:57:10.889429 28949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987: Bootstrap starting.
I20260929 02:57:10.892385 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:10.892921 28949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:10.894088 28949 log.cc:824] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:10.897261 28949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987: No bootstrap required, opened a new log
I20260929 02:57:10.903409 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:10.897139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48917 (local address 127.0.0.1:41371)
0929 02:57:10.897317 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:10.897322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:10.897585 (+ 263us) server_negotiation.cc:407] Connection header received
0929 02:57:10.897720 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:10.897726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:10.897800 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:10.897907 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:10.898368 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:10.899511 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:10.900907 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:10.901565 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:10.901682 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:10.902672 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:10.902678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:10.902683 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:10.902704 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:10.902768 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:10.902788 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:10.902899 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:10.903022 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 02:57:10.903077 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:57:10.906322 28949 raft_consensus.cc:348] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95f13efd99246769623f4f32dea0987" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260929 02:57:10.906641 28949 raft_consensus.cc:374] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:10.906710 28949 raft_consensus.cc:729] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f13efd99246769623f4f32dea0987, State: Initialized, Role: FOLLOWER
I20260929 02:57:10.907184 28949 consensus_queue.cc:260] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95f13efd99246769623f4f32dea0987" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260929 02:57:10.907371 28949 raft_consensus.cc:388] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:10.907459 28949 raft_consensus.cc:482] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:10.907580 28949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:10.909695 28949 raft_consensus.cc:504] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95f13efd99246769623f4f32dea0987" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260929 02:57:10.910200 28949 leader_election.cc:302] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c95f13efd99246769623f4f32dea0987; no voters:
I20260929 02:57:10.910665 28949 leader_election.cc:288] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:10.910797 28959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:10.911397 28959 raft_consensus.cc:686] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [term 1 LEADER]: Becoming Leader. State: Replica: c95f13efd99246769623f4f32dea0987, State: Running, Role: LEADER
I20260929 02:57:10.911922 28959 consensus_queue.cc:237] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95f13efd99246769623f4f32dea0987" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260929 02:57:10.912511 28949 sys_catalog.cc:564] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:10.914764 28961 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c95f13efd99246769623f4f32dea0987. Latest consensus state: current_term: 1 leader_uuid: "c95f13efd99246769623f4f32dea0987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95f13efd99246769623f4f32dea0987" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } } }
I20260929 02:57:10.916309 28961 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:10.915407 28960 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c95f13efd99246769623f4f32dea0987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95f13efd99246769623f4f32dea0987" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } } }
I20260929 02:57:10.921845 28960 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987 [sys.catalog]: This master's current role is: LEADER
W20260929 02:57:10.928973 28975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:57:10.929104 28975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:57:10.929231 28969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:10.931384 28969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:10.940213 28969 catalog_manager.cc:1347] Generated new cluster ID: 04818ee08d6040a88330d80310b85b38
I20260929 02:57:10.940342 28969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:10.953425 28969 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:10.954766 28969 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:10.962121 28969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c95f13efd99246769623f4f32dea0987: Generated new TSK 0
I20260929 02:57:10.963238 28969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:10.977972 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28882
W20260929 02:57:10.979130 28944 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42425
W20260929 02:57:10.979949 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28882: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:10.980131 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28882
W20260929 02:57:10.993311 28867 connection.cc:535] client connection to 127.0.0.1:41371 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41371): Connection reset by peer (error 104)
2026-09-29T02:57:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1770 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:11 dist-test-slave-1dkt krb5kdc[28980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:11 dist-test-slave-1dkt krb5kdc[28980](info): set up 2 sockets
Sep 29 02:57:11 dist-test-slave-1dkt krb5kdc[28980](info): commencing operation
krb5kdc: starting...
W20260929 02:57:11.992089 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:12 dist-test-slave-1dkt krb5kdc[28980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:12.222574 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:12.536821 28993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:12.537109 28993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:12.537228 28993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:12.544531 28993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:12.544667 28993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:12.544745 28993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:12.544831 28993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:12.544891 28993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:12.555109 28993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39691
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:12.556416 28993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:12.557682 28993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:12.568795 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:12.568765 29001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:12.568841 28999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:12.569779 28993 server_base.cc:1034] running on GCE node
I20260929 02:57:12.570477 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:12.571771 28993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:12.572965 28993 hybrid_clock.cc:648] HybridClock initialized: now 1790650632572919 us; error 40 us; skew 500 ppm
I20260929 02:57:12.573333 28993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:12 dist-test-slave-1dkt krb5kdc[28980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:12.579519 28993 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:12.581069 28993 webserver.cc:460] Webserver started at http://127.0.0.1:37499/ using document root <none> and password file <none>
I20260929 02:57:12.581725 28993 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:12.581841 28993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:12.582129 28993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:12.585134 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "f2737de745c14dc39eeb854e2d2193e2"
format_stamp: "Formatted at 2026-09-29 02:57:12 on dist-test-slave-1dkt"
server_key: "c74a803d216204a18109d750e29b3325"
server_key_iv: "4f0c0eb8a6f7e4f9fdf6caf5c071f852"
server_key_version: "encryptionkey@0"
I20260929 02:57:12.585879 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "f2737de745c14dc39eeb854e2d2193e2"
format_stamp: "Formatted at 2026-09-29 02:57:12 on dist-test-slave-1dkt"
server_key: "c74a803d216204a18109d750e29b3325"
server_key_iv: "4f0c0eb8a6f7e4f9fdf6caf5c071f852"
server_key_version: "encryptionkey@0"
I20260929 02:57:12.590939 28993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:57:12.594300 29008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:12.595551 28993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:12.595698 28993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "f2737de745c14dc39eeb854e2d2193e2"
format_stamp: "Formatted at 2026-09-29 02:57:12 on dist-test-slave-1dkt"
server_key: "c74a803d216204a18109d750e29b3325"
server_key_iv: "4f0c0eb8a6f7e4f9fdf6caf5c071f852"
server_key_version: "encryptionkey@0"
I20260929 02:57:12.595889 28993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:12.601358 28993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:12.605232 28993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:12.605486 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:12.622205 28993 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39691
I20260929 02:57:12.622241 29059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39691 every 8 connection(s)
I20260929 02:57:12.623541 28993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:12.624812 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 28993
I20260929 02:57:12.624977 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:12.625505 8420 external_mini_cluster.cc:1428] Setting key ed60aa170b482e8bab23fd7ac8b1190f
I20260929 02:57:12.628051 29060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:12.633045 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:12.627276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46517 (local address 127.0.0.1:39691)
0929 02:57:12.628105 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:12.628120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:12.628169 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:57:12.629768 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:12.629790 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:12.630117 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:12.630486 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:12.631466 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:12.631471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:12.631488 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:12.631503 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:12.631593 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:12.631616 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:12.631819 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:12.631976 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 02:57:12.632159 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":330,"threads_started":1}
I20260929 02:57:12.636492 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:12.638886 29060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2: Bootstrap starting.
I20260929 02:57:12.642191 29060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:12.642946 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:12.640648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40575 (local address 127.0.0.1:39691)
0929 02:57:12.640822 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:12.640827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:12.641109 (+ 282us) server_negotiation.cc:407] Connection header received
0929 02:57:12.641283 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:12.641286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:12.641330 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:12.641395 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:12.642349 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:12.642355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:12.642358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:12.642372 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:12.642411 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:12.642426 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:12.642510 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:12.642658 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 02:57:12.642702 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:57:12.643504 29060 log.cc:824] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:12.646236 29060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2: No bootstrap required, opened a new log
I20260929 02:57:12.653479 29060 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2737de745c14dc39eeb854e2d2193e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260929 02:57:12.653846 29060 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:12.653939 29060 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2737de745c14dc39eeb854e2d2193e2, State: Initialized, Role: FOLLOWER
I20260929 02:57:12.654484 29060 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2737de745c14dc39eeb854e2d2193e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260929 02:57:12.654690 29060 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:12.654810 29060 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:12.654958 29060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:12.657274 29060 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2737de745c14dc39eeb854e2d2193e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260929 02:57:12.657732 29060 leader_election.cc:302] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2737de745c14dc39eeb854e2d2193e2; no voters:
I20260929 02:57:12.658262 29060 leader_election.cc:288] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:12.658384 29070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:12.659006 29070 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [term 1 LEADER]: Becoming Leader. State: Replica: f2737de745c14dc39eeb854e2d2193e2, State: Running, Role: LEADER
I20260929 02:57:12.659559 29070 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2737de745c14dc39eeb854e2d2193e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260929 02:57:12.660334 29060 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:12.662304 29071 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2737de745c14dc39eeb854e2d2193e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2737de745c14dc39eeb854e2d2193e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } } }
I20260929 02:57:12.662328 29072 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2737de745c14dc39eeb854e2d2193e2. Latest consensus state: current_term: 1 leader_uuid: "f2737de745c14dc39eeb854e2d2193e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2737de745c14dc39eeb854e2d2193e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } } }
I20260929 02:57:12.662768 29072 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:12.662765 29071 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:12.664759 29076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:12.668970 29076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:12.681597 29076 catalog_manager.cc:1347] Generated new cluster ID: 4e84dd8524b549b3b3bf813c45106e5f
I20260929 02:57:12.681713 29076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:12.706935 29076 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:12.707381 29076 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:12.717976 29076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2737de745c14dc39eeb854e2d2193e2: Generated new TSK 0
I20260929 02:57:12.718765 29076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:12.723296 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28993
W20260929 02:57:12.724706 29055 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46517
W20260929 02:57:12.725369 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 28993: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:12.725517 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 28993
2026-09-29T02:57:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1745 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:12 dist-test-slave-1dkt krb5kdc[29091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:12 dist-test-slave-1dkt krb5kdc[29091](info): set up 2 sockets
Sep 29 02:57:12 dist-test-slave-1dkt krb5kdc[29091](info): commencing operation
krb5kdc: starting...
W20260929 02:57:13.728171 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:13 dist-test-slave-1dkt krb5kdc[29091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:13.959959 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:14.282222 29104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:14.282507 29104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:14.282629 29104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:14.290000 29104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:14.290123 29104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:14.290197 29104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:14.290280 29104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:14.290338 29104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:14.300344 29104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:14.301632 29104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:14.302923 29104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:14.314214 29109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:14.314180 29110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:14.314147 29112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:14.314965 29104 server_base.cc:1034] running on GCE node
I20260929 02:57:14.315706 29104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:14.317061 29104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:14.318279 29104 hybrid_clock.cc:648] HybridClock initialized: now 1790650634318235 us; error 39 us; skew 500 ppm
I20260929 02:57:14.318589 29104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:14 dist-test-slave-1dkt krb5kdc[29091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:14.321602 29104 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:14.323050 29104 webserver.cc:460] Webserver started at http://127.0.0.1:41695/ using document root <none> and password file <none>
I20260929 02:57:14.323694 29104 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:14.323797 29104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:14.324146 29104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:14.326967 29104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "6c13d7ba10ea48598ca4d959c308e48e"
format_stamp: "Formatted at 2026-09-29 02:57:14 on dist-test-slave-1dkt"
server_key: "6e1bb79f29e05e41105f3a6c143314da"
server_key_iv: "2c69db1c5c9793d3e51970a386596add"
server_key_version: "encryptionkey@0"
I20260929 02:57:14.327674 29104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "6c13d7ba10ea48598ca4d959c308e48e"
format_stamp: "Formatted at 2026-09-29 02:57:14 on dist-test-slave-1dkt"
server_key: "6e1bb79f29e05e41105f3a6c143314da"
server_key_iv: "2c69db1c5c9793d3e51970a386596add"
server_key_version: "encryptionkey@0"
I20260929 02:57:14.332566 29104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:57:14.335758 29119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:14.336997 29104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:57:14.337153 29104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "6c13d7ba10ea48598ca4d959c308e48e"
format_stamp: "Formatted at 2026-09-29 02:57:14 on dist-test-slave-1dkt"
server_key: "6e1bb79f29e05e41105f3a6c143314da"
server_key_iv: "2c69db1c5c9793d3e51970a386596add"
server_key_version: "encryptionkey@0"
I20260929 02:57:14.337312 29104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:14.355645 29104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:14.359455 29104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:14.359709 29104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:14.376616 29104 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42695
I20260929 02:57:14.376662 29170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42695 every 8 connection(s)
I20260929 02:57:14.377918 29104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:14.381445 29171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:14.382515 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29104
I20260929 02:57:14.382668 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:14.383096 8420 external_mini_cluster.cc:1428] Setting key 44319db503ca746b3a7510463e193ef0
I20260929 02:57:14.394923 29171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e: Bootstrap starting.
Sep 29 02:57:14 dist-test-slave-1dkt krb5kdc[29091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:57:14.398645 29171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:14.399711 29171 log.cc:824] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:14.402563 29171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e: No bootstrap required, opened a new log
I20260929 02:57:14.405906 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:14.384853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59965 (local address 127.0.0.1:42695)
0929 02:57:14.385643 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:14.385658 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:14.385710 (+ 52us) server_negotiation.cc:407] Connection header received
0929 02:57:14.387530 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:14.387562 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:14.388116 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:14.388534 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:14.389742 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:14.391800 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:14.393192 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:14.394189 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:14.394352 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:14.399026 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:14.399036 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:14.399050 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:14.399092 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:14.401922 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:14.402772 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:14.402779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:14.402785 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:14.402946 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:14.403361 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:14.403364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:14.403366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:14.403865 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:14.404107 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:14.404548 (+ 441us) server_negotiation.cc:299] Negotiation successful
0929 02:57:14.404900 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":310,"threads_started":1}
I20260929 02:57:14.409376 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:14.410662 29171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c13d7ba10ea48598ca4d959c308e48e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260929 02:57:14.410974 29171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:14.411060 29171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c13d7ba10ea48598ca4d959c308e48e, State: Initialized, Role: FOLLOWER
I20260929 02:57:14.411578 29171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c13d7ba10ea48598ca4d959c308e48e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260929 02:57:14.411795 29171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:14.411959 29171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:14.412143 29171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:14.414633 29171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c13d7ba10ea48598ca4d959c308e48e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260929 02:57:14.415083 29171 leader_election.cc:302] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c13d7ba10ea48598ca4d959c308e48e; no voters:
I20260929 02:57:14.415655 29171 leader_election.cc:288] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:14.415759 29181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:14.417119 29181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [term 1 LEADER]: Becoming Leader. State: Replica: 6c13d7ba10ea48598ca4d959c308e48e, State: Running, Role: LEADER
I20260929 02:57:14.417860 29171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:14.417735 29181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c13d7ba10ea48598ca4d959c308e48e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260929 02:57:14.425160 29183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c13d7ba10ea48598ca4d959c308e48e. Latest consensus state: current_term: 1 leader_uuid: "6c13d7ba10ea48598ca4d959c308e48e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c13d7ba10ea48598ca4d959c308e48e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } } }
I20260929 02:57:14.425571 29183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:14.426192 29182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c13d7ba10ea48598ca4d959c308e48e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c13d7ba10ea48598ca4d959c308e48e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } } }
I20260929 02:57:14.426481 29182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:14.428323 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:14.413449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36841 (local address 127.0.0.1:42695)
0929 02:57:14.413624 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:14.413630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:14.413998 (+ 368us) server_negotiation.cc:407] Connection header received
0929 02:57:14.414154 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:14.414159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:14.414227 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:14.414305 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:14.415722 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:14.416834 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:14.418200 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:14.418675 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:14.418785 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:14.421182 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:14.421188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:14.421196 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:14.421234 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:14.424120 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:14.424963 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:14.424967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:14.424970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:14.425108 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:14.425552 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:14.425571 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:14.425574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:14.426025 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:14.426268 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:14.427634 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0929 02:57:14.427925 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:57:14.430282 29189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:14.434517 29189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:14.443284 29189 catalog_manager.cc:1347] Generated new cluster ID: 69c72e102d454d91977983a9f24c62fc
I20260929 02:57:14.443391 29189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:14.469697 29189 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:14.471031 29189 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:14.480664 29189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c13d7ba10ea48598ca4d959c308e48e: Generated new TSK 0
I20260929 02:57:14.481500 29189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:14.490192 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29104
I20260929 02:57:14.491309 29166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:14.583250 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29104
2026-09-29T02:57:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1858 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:14 dist-test-slave-1dkt krb5kdc[29204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:14 dist-test-slave-1dkt krb5kdc[29204](info): set up 2 sockets
Sep 29 02:57:14 dist-test-slave-1dkt krb5kdc[29204](info): commencing operation
krb5kdc: starting...
W20260929 02:57:15.600231 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:15 dist-test-slave-1dkt krb5kdc[29204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:15.837507 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:16.161991 29217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:16.162271 29217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:16.162401 29217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:16.169497 29217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:16.169600 29217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:16.169677 29217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:16.169750 29217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:16.169806 29217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:16.179543 29217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:16.180823 29217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:16.182125 29217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:16.193078 29222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:16.193203 29223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:16.193176 29225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:16.194056 29217 server_base.cc:1034] running on GCE node
I20260929 02:57:16.194787 29217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:16.196213 29217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:16.197407 29217 hybrid_clock.cc:648] HybridClock initialized: now 1790650636197362 us; error 35 us; skew 500 ppm
I20260929 02:57:16.197724 29217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:16 dist-test-slave-1dkt krb5kdc[29204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:16.200851 29217 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:16.202283 29217 webserver.cc:460] Webserver started at http://127.0.0.1:34847/ using document root <none> and password file <none>
I20260929 02:57:16.202942 29217 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:16.203056 29217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:16.203336 29217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:16.206243 29217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "7e16a1f4d8974087aba45ce2be022288"
format_stamp: "Formatted at 2026-09-29 02:57:16 on dist-test-slave-1dkt"
server_key: "c3a85aec8ec89fce4d4a9c4119c1e948"
server_key_iv: "a832712266307aca32d63dd51dd78d44"
server_key_version: "encryptionkey@0"
I20260929 02:57:16.206972 29217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "7e16a1f4d8974087aba45ce2be022288"
format_stamp: "Formatted at 2026-09-29 02:57:16 on dist-test-slave-1dkt"
server_key: "c3a85aec8ec89fce4d4a9c4119c1e948"
server_key_iv: "a832712266307aca32d63dd51dd78d44"
server_key_version: "encryptionkey@0"
I20260929 02:57:16.212133 29217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:57:16.215505 29232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:16.216847 29217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:16.217005 29217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "7e16a1f4d8974087aba45ce2be022288"
format_stamp: "Formatted at 2026-09-29 02:57:16 on dist-test-slave-1dkt"
server_key: "c3a85aec8ec89fce4d4a9c4119c1e948"
server_key_iv: "a832712266307aca32d63dd51dd78d44"
server_key_version: "encryptionkey@0"
I20260929 02:57:16.217175 29217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:16.235481 29217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:16.239559 29217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:16.239822 29217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:16.256431 29217 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35093
I20260929 02:57:16.256464 29283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35093 every 8 connection(s)
I20260929 02:57:16.257757 29217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:16.260499 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29217
I20260929 02:57:16.260643 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:16.261057 8420 external_mini_cluster.cc:1428] Setting key e98270c6a4e2b5e46760b66b33ebc362
I20260929 02:57:16.261399 29284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:16.275484 29287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:16.262720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60331 (local address 127.0.0.1:35093)
0929 02:57:16.263481 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:16.263496 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:16.263554 (+ 58us) server_negotiation.cc:407] Connection header received
0929 02:57:16.265917 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:16.265945 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:16.266450 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:16.266901 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:16.267425 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:16.269398 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:16.270671 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:16.271650 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:16.271832 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:16.272891 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:16.272901 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:16.272919 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:16.273631 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:16.273727 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:16.273756 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:16.273861 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:16.274026 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 02:57:16.274279 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":308,"threads_started":1}
I20260929 02:57:16.275671 29284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288: Bootstrap starting.
I20260929 02:57:16.278787 29284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:16.278973 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:16.279793 29284 log.cc:824] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:16.282549 29284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288: No bootstrap required, opened a new log
I20260929 02:57:16.289548 29287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:16.283123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39437 (local address 127.0.0.1:35093)
0929 02:57:16.283326 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:16.283333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:16.283811 (+ 478us) server_negotiation.cc:407] Connection header received
0929 02:57:16.283978 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:16.283983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:16.284056 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:16.284173 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:16.284609 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:16.285701 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:16.286969 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:16.287674 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:16.287796 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:16.288816 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:16.288822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:16.288827 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:16.288851 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:16.288895 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:16.288913 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:16.289012 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:16.289129 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 02:57:16.289213 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:57:16.291495 29284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e16a1f4d8974087aba45ce2be022288" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35093 } }
I20260929 02:57:16.291831 29284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:16.291941 29284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e16a1f4d8974087aba45ce2be022288, State: Initialized, Role: FOLLOWER
I20260929 02:57:16.292510 29284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e16a1f4d8974087aba45ce2be022288" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35093 } }
I20260929 02:57:16.292722 29284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:16.292846 29284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:16.292990 29284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:16.295292 29284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e16a1f4d8974087aba45ce2be022288" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35093 } }
I20260929 02:57:16.295737 29284 leader_election.cc:302] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e16a1f4d8974087aba45ce2be022288; no voters:
I20260929 02:57:16.296311 29284 leader_election.cc:288] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:16.296419 29294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:16.297027 29294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [term 1 LEADER]: Becoming Leader. State: Replica: 7e16a1f4d8974087aba45ce2be022288, State: Running, Role: LEADER
I20260929 02:57:16.297597 29294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e16a1f4d8974087aba45ce2be022288" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35093 } }
I20260929 02:57:16.298264 29284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:16.300493 29295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e16a1f4d8974087aba45ce2be022288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e16a1f4d8974087aba45ce2be022288" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35093 } } }
I20260929 02:57:16.300796 29295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:16.302179 29302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:16.302250 29296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e16a1f4d8974087aba45ce2be022288. Latest consensus state: current_term: 1 leader_uuid: "7e16a1f4d8974087aba45ce2be022288" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e16a1f4d8974087aba45ce2be022288" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35093 } } }
I20260929 02:57:16.302515 29296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:16.306291 29302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:16.314746 29302 catalog_manager.cc:1347] Generated new cluster ID: c8c9c23394e9430c845cc77699d4b57f
I20260929 02:57:16.314901 29302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:16.360554 29302 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:16.361940 29302 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:16.372191 29302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e16a1f4d8974087aba45ce2be022288: Generated new TSK 0
I20260929 02:57:16.373025 29302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:16.379012 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29217
W20260929 02:57:16.380096 29279 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60331
W20260929 02:57:16.380702 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29217: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:16.380853 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29217
2026-09-29T02:57:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1796 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:16 dist-test-slave-1dkt krb5kdc[29315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:16 dist-test-slave-1dkt krb5kdc[29315](info): set up 2 sockets
Sep 29 02:57:16 dist-test-slave-1dkt krb5kdc[29315](info): commencing operation
krb5kdc: starting...
W20260929 02:57:17.396188 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:17 dist-test-slave-1dkt krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:17.625356 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:17.939616 29328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:17.939917 29328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:17.940052 29328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:17.947515 29328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:17.947649 29328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:17.947729 29328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:17.947813 29328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:17.947889 29328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:17.957756 29328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:17.959000 29328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:17.960266 29328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:17.971424 29333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:17.971480 29334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:17.971848 29336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:17.972180 29328 server_base.cc:1034] running on GCE node
I20260929 02:57:17.972903 29328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:17.974241 29328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:17.975445 29328 hybrid_clock.cc:648] HybridClock initialized: now 1790650637975395 us; error 42 us; skew 500 ppm
I20260929 02:57:17.975796 29328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:17 dist-test-slave-1dkt krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650637, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:17.983080 29328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:17.984676 29328 webserver.cc:460] Webserver started at http://127.0.0.1:36117/ using document root <none> and password file <none>
I20260929 02:57:17.985330 29328 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:17.985430 29328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:17.985687 29328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:17.988624 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "c0915f57c628436f9d27a9fc2ecf7598"
format_stamp: "Formatted at 2026-09-29 02:57:17 on dist-test-slave-1dkt"
server_key: "9b21169f3b76deef559a80c909f2a47d"
server_key_iv: "79f3d9da179e225d7a571aeb4dc6ccb6"
server_key_version: "encryptionkey@0"
I20260929 02:57:17.989331 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "c0915f57c628436f9d27a9fc2ecf7598"
format_stamp: "Formatted at 2026-09-29 02:57:17 on dist-test-slave-1dkt"
server_key: "9b21169f3b76deef559a80c909f2a47d"
server_key_iv: "79f3d9da179e225d7a571aeb4dc6ccb6"
server_key_version: "encryptionkey@0"
I20260929 02:57:17.994329 29328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:57:17.997669 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:17.998999 29328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:57:17.999140 29328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "c0915f57c628436f9d27a9fc2ecf7598"
format_stamp: "Formatted at 2026-09-29 02:57:17 on dist-test-slave-1dkt"
server_key: "9b21169f3b76deef559a80c909f2a47d"
server_key_iv: "79f3d9da179e225d7a571aeb4dc6ccb6"
server_key_version: "encryptionkey@0"
I20260929 02:57:17.999310 29328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:18.004664 29328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:18.008721 29328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:18.008978 29328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:18.026486 29328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44273
I20260929 02:57:18.026521 29394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44273 every 8 connection(s)
I20260929 02:57:18.027829 29328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:18.031436 29395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:18.037142 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29328
I20260929 02:57:18.037315 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:18.037868 8420 external_mini_cluster.cc:1428] Setting key b10b3cb5115cf4c57fb0aae323d88e57
I20260929 02:57:18.042364 29395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598: Bootstrap starting.
I20260929 02:57:18.045571 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:18.039680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47299 (local address 127.0.0.1:44273)
0929 02:57:18.040553 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:18.040564 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:18.040604 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:57:18.042324 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:18.042346 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:18.042717 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:18.043032 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:18.044087 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:18.044093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:18.044105 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:18.044120 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:18.044203 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:18.044227 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:18.044389 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:18.044506 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 02:57:18.044671 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":426,"threads_started":1}
I20260929 02:57:18.045805 29395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:18.046850 29395 log.cc:824] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:18.049082 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:18.049836 29395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598: No bootstrap required, opened a new log
I20260929 02:57:18.056319 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:18.053974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33285 (local address 127.0.0.1:44273)
0929 02:57:18.054112 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:18.054116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:18.054489 (+ 373us) server_negotiation.cc:407] Connection header received
0929 02:57:18.054659 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:18.054663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:18.054715 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:18.054798 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:18.055684 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:18.055689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:18.055692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:18.055707 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:18.055749 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:18.055764 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:18.055857 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:18.056005 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 02:57:18.056052 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 02:57:18.059070 29395 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0915f57c628436f9d27a9fc2ecf7598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44273 } }
I20260929 02:57:18.059499 29395 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:18.059583 29395 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0915f57c628436f9d27a9fc2ecf7598, State: Initialized, Role: FOLLOWER
I20260929 02:57:18.060179 29395 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0915f57c628436f9d27a9fc2ecf7598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44273 } }
I20260929 02:57:18.060389 29395 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:18.060511 29395 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:18.060664 29395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:18.062732 29395 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0915f57c628436f9d27a9fc2ecf7598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44273 } }
I20260929 02:57:18.063225 29395 leader_election.cc:302] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0915f57c628436f9d27a9fc2ecf7598; no voters:
I20260929 02:57:18.063733 29395 leader_election.cc:288] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:18.063848 29405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:18.064536 29405 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [term 1 LEADER]: Becoming Leader. State: Replica: c0915f57c628436f9d27a9fc2ecf7598, State: Running, Role: LEADER
I20260929 02:57:18.065065 29405 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0915f57c628436f9d27a9fc2ecf7598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44273 } }
I20260929 02:57:18.065768 29395 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:18.067622 29406 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0915f57c628436f9d27a9fc2ecf7598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0915f57c628436f9d27a9fc2ecf7598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44273 } } }
I20260929 02:57:18.067982 29407 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0915f57c628436f9d27a9fc2ecf7598. Latest consensus state: current_term: 1 leader_uuid: "c0915f57c628436f9d27a9fc2ecf7598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0915f57c628436f9d27a9fc2ecf7598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44273 } } }
I20260929 02:57:18.068658 29406 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:18.068730 29407 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:18.070179 29413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:18.074344 29413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:18.082221 29413 catalog_manager.cc:1347] Generated new cluster ID: e7e4a77e01ea476d9e8d328c195d3dbb
I20260929 02:57:18.082332 29413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:18.092599 29413 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:18.092998 29413 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:18.099453 29413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0915f57c628436f9d27a9fc2ecf7598: Generated new TSK 0
I20260929 02:57:18.100274 29413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:18.111334 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29328
W20260929 02:57:18.112672 29390 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47299
W20260929 02:57:18.113363 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29328: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:18.113507 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29328
2026-09-29T02:57:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1731 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:18 dist-test-slave-1dkt krb5kdc[29426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:18 dist-test-slave-1dkt krb5kdc[29426](info): set up 2 sockets
Sep 29 02:57:18 dist-test-slave-1dkt krb5kdc[29426](info): commencing operation
krb5kdc: starting...
W20260929 02:57:19.124336 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:19 dist-test-slave-1dkt krb5kdc[29426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:19.350447 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:19.658865 29439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:19.659152 29439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:19.659283 29439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:19.666545 29439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:19.666668 29439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:19.666729 29439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:19.666810 29439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:19.666864 29439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:19.676683 29439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:19.677891 29439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:19.679137 29439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:19.690006 29444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:19.690047 29445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:19.690073 29447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:19.691000 29439 server_base.cc:1034] running on GCE node
I20260929 02:57:19.691726 29439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:19.693069 29439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:19.694264 29439 hybrid_clock.cc:648] HybridClock initialized: now 1790650639694221 us; error 38 us; skew 500 ppm
I20260929 02:57:19.694588 29439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:19 dist-test-slave-1dkt krb5kdc[29426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:19.697512 29439 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:19.698952 29439 webserver.cc:460] Webserver started at http://127.0.0.1:41167/ using document root <none> and password file <none>
I20260929 02:57:19.699604 29439 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:19.699728 29439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:19.700073 29439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:19.702970 29439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "0853793eba7c4fdf822b49e867f3219f"
format_stamp: "Formatted at 2026-09-29 02:57:19 on dist-test-slave-1dkt"
server_key: "76124e90a6e3ae70d1756c833879c476"
server_key_iv: "12c2b13413d319861b544e3f12869876"
server_key_version: "encryptionkey@0"
I20260929 02:57:19.703683 29439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "0853793eba7c4fdf822b49e867f3219f"
format_stamp: "Formatted at 2026-09-29 02:57:19 on dist-test-slave-1dkt"
server_key: "76124e90a6e3ae70d1756c833879c476"
server_key_iv: "12c2b13413d319861b544e3f12869876"
server_key_version: "encryptionkey@0"
I20260929 02:57:19.708534 29439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 02:57:19.711684 29454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:19.712929 29439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 02:57:19.713097 29439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "0853793eba7c4fdf822b49e867f3219f"
format_stamp: "Formatted at 2026-09-29 02:57:19 on dist-test-slave-1dkt"
server_key: "76124e90a6e3ae70d1756c833879c476"
server_key_iv: "12c2b13413d319861b544e3f12869876"
server_key_version: "encryptionkey@0"
I20260929 02:57:19.713280 29439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:19.735383 29439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:19.739077 29439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:19.739334 29439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:19.756460 29439 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38369
I20260929 02:57:19.756482 29505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38369 every 8 connection(s)
I20260929 02:57:19.757791 29439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:19.760804 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29439
I20260929 02:57:19.760969 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:19.761535 8420 external_mini_cluster.cc:1428] Setting key 5c3864ba8cc9845afb5f46a91253ee5c
I20260929 02:57:19.761861 29506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:19.775671 29506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f: Bootstrap starting.
Sep 29 02:57:19 dist-test-slave-1dkt krb5kdc[29426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:57:19.778767 29506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:19.779788 29506 log.cc:824] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:19.782425 29506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f: No bootstrap required, opened a new log
I20260929 02:57:19.783767 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:19.763226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:47339 (local address 127.0.0.1:38369)
0929 02:57:19.763983 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:19.763994 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:19.764033 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:57:19.766275 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:19.766297 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:19.766689 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:19.767001 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:19.768718 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:19.769972 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:19.771679 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:19.772401 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:19.772538 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:19.777033 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:19.777041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:19.777057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:19.777094 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:19.779905 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:19.780657 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:19.780664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:19.780669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:19.780842 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:19.781283 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:19.781286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:19.781289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:19.781776 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:19.782052 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:19.782548 (+ 496us) server_negotiation.cc:299] Negotiation successful
0929 02:57:19.782845 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":312,"threads_started":1}
I20260929 02:57:19.787575 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:19.790576 29506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0853793eba7c4fdf822b49e867f3219f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260929 02:57:19.790967 29506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:19.791057 29506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0853793eba7c4fdf822b49e867f3219f, State: Initialized, Role: FOLLOWER
I20260929 02:57:19.791749 29506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0853793eba7c4fdf822b49e867f3219f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260929 02:57:19.792061 29506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:19.792196 29506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:19.792356 29506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:19.794450 29506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0853793eba7c4fdf822b49e867f3219f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260929 02:57:19.794929 29506 leader_election.cc:302] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0853793eba7c4fdf822b49e867f3219f; no voters:
I20260929 02:57:19.795511 29506 leader_election.cc:288] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:19.795601 29516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:19.796715 29516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [term 1 LEADER]: Becoming Leader. State: Replica: 0853793eba7c4fdf822b49e867f3219f, State: Running, Role: LEADER
I20260929 02:57:19.797374 29516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0853793eba7c4fdf822b49e867f3219f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260929 02:57:19.797927 29506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:19.800316 29517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0853793eba7c4fdf822b49e867f3219f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0853793eba7c4fdf822b49e867f3219f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } } }
I20260929 02:57:19.800243 29518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0853793eba7c4fdf822b49e867f3219f. Latest consensus state: current_term: 1 leader_uuid: "0853793eba7c4fdf822b49e867f3219f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0853793eba7c4fdf822b49e867f3219f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } } }
I20260929 02:57:19.800922 29518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:19.800922 29517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:19.801759 29523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:19.807840 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:19.791659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:49459 (local address 127.0.0.1:38369)
0929 02:57:19.791802 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:19.791808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:19.792260 (+ 452us) server_negotiation.cc:407] Connection header received
0929 02:57:19.792412 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:19.792418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:19.792489 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:19.792574 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:19.793847 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:19.794566 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:19.795827 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:19.796487 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:19.796602 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:19.799451 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:19.799457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:19.799465 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:19.799500 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:19.803411 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:19.804469 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:19.804475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:19.804479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:19.804629 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:19.805199 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:19.805205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:19.805209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:19.805691 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:19.807067 (+ 1376us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:19.807252 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 02:57:19.807449 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:57:19.819268 29523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:19.828955 29523 catalog_manager.cc:1347] Generated new cluster ID: 254a0dd237474568845a6a57e7fcd046
I20260929 02:57:19.829067 29523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:19.857301 29523 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:19.859138 29523 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:19.867326 29523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0853793eba7c4fdf822b49e867f3219f: Generated new TSK 0
I20260929 02:57:19.868395 29523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:19.878600 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29439
I20260929 02:57:19.879719 29501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:19.971675 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29439
W20260929 02:57:19.985170 29424 connection.cc:535] client connection to 127.0.0.1:38369 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38369): Connection reset by peer (error 104)
2026-09-29T02:57:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1858 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:20 dist-test-slave-1dkt krb5kdc[29538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:20 dist-test-slave-1dkt krb5kdc[29538](info): set up 2 sockets
Sep 29 02:57:20 dist-test-slave-1dkt krb5kdc[29538](info): commencing operation
krb5kdc: starting...
W20260929 02:57:20.980239 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:21 dist-test-slave-1dkt krb5kdc[29538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:21.217200 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:21.536293 29551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:21.536572 29551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:21.536710 29551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:21.544248 29551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:21.544355 29551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:21.544425 29551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:21.544502 29551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:21.544557 29551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:21.554836 29551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:21.556362 29551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:21.557682 29551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:21.569125 29556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:21.569164 29559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:21.569183 29557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:21.570289 29551 server_base.cc:1034] running on GCE node
I20260929 02:57:21.571118 29551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:21.572620 29551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:21.573848 29551 hybrid_clock.cc:648] HybridClock initialized: now 1790650641573793 us; error 52 us; skew 500 ppm
I20260929 02:57:21.574237 29551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:21 dist-test-slave-1dkt krb5kdc[29538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:21.577527 29551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:21.578958 29551 webserver.cc:460] Webserver started at http://127.0.0.1:37683/ using document root <none> and password file <none>
I20260929 02:57:21.579675 29551 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:21.579793 29551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:21.580103 29551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:21.582970 29551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "3dd1868fd79f4536a1a70cc0c34ebef3"
format_stamp: "Formatted at 2026-09-29 02:57:21 on dist-test-slave-1dkt"
server_key: "806556f35460f46765ab80fb0eefe46d"
server_key_iv: "099414ff534f5371614a54cf7d4388fd"
server_key_version: "encryptionkey@0"
I20260929 02:57:21.583698 29551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "3dd1868fd79f4536a1a70cc0c34ebef3"
format_stamp: "Formatted at 2026-09-29 02:57:21 on dist-test-slave-1dkt"
server_key: "806556f35460f46765ab80fb0eefe46d"
server_key_iv: "099414ff534f5371614a54cf7d4388fd"
server_key_version: "encryptionkey@0"
I20260929 02:57:21.588696 29551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:57:21.591854 29566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:21.593092 29551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 02:57:21.593236 29551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "3dd1868fd79f4536a1a70cc0c34ebef3"
format_stamp: "Formatted at 2026-09-29 02:57:21 on dist-test-slave-1dkt"
server_key: "806556f35460f46765ab80fb0eefe46d"
server_key_iv: "099414ff534f5371614a54cf7d4388fd"
server_key_version: "encryptionkey@0"
I20260929 02:57:21.593392 29551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:21.614610 29551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:21.618381 29551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:21.618628 29551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:21.635057 29551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35579
I20260929 02:57:21.635087 29617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35579 every 8 connection(s)
I20260929 02:57:21.636396 29551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:21.639454 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29551
I20260929 02:57:21.639626 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:21.640198 8420 external_mini_cluster.cc:1428] Setting key aa4f7cd97e4ade4d4f81aad124c5ce47
I20260929 02:57:21.640199 29618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:21.653802 29618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3: Bootstrap starting.
I20260929 02:57:21.653805 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:21.641840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:52605 (local address 127.0.0.1:35579)
0929 02:57:21.642544 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:21.642554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:21.642610 (+ 56us) server_negotiation.cc:407] Connection header received
0929 02:57:21.644675 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:21.644697 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:21.645101 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:21.645425 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:21.645997 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:21.647324 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:21.649225 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:21.650002 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:21.650163 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:21.651489 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:21.651497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:21.651512 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:21.652137 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:21.652221 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:21.652249 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:21.652367 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:21.652617 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 02:57:21.652812 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":309,"threads_started":1}
I20260929 02:57:21.657320 29618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:21.657560 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:21.658548 29618 log.cc:824] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:21.661788 29618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3: No bootstrap required, opened a new log
I20260929 02:57:21.669087 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:21.662389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:39395 (local address 127.0.0.1:35579)
0929 02:57:21.662574 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:21.662579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:21.663067 (+ 488us) server_negotiation.cc:407] Connection header received
0929 02:57:21.663255 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:21.663259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:21.663319 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:21.663391 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:21.663976 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:21.664761 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:21.666564 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:21.667052 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:21.667141 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:21.668366 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:21.668370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:21.668374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:21.668392 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:21.668432 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:21.668449 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:21.668524 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:21.668738 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 02:57:21.668789 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:57:21.671222 29618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260929 02:57:21.671569 29618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:21.671665 29618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dd1868fd79f4536a1a70cc0c34ebef3, State: Initialized, Role: FOLLOWER
I20260929 02:57:21.672238 29618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260929 02:57:21.672446 29618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:21.672572 29618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:21.672724 29618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:21.674800 29618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260929 02:57:21.675279 29618 leader_election.cc:302] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dd1868fd79f4536a1a70cc0c34ebef3; no voters:
I20260929 02:57:21.675808 29618 leader_election.cc:288] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:21.675977 29628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:21.677865 29618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:21.680236 29628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [term 1 LEADER]: Becoming Leader. State: Replica: 3dd1868fd79f4536a1a70cc0c34ebef3, State: Running, Role: LEADER
I20260929 02:57:21.680747 29628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } }
I20260929 02:57:21.683521 29629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } } }
I20260929 02:57:21.683496 29630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dd1868fd79f4536a1a70cc0c34ebef3. Latest consensus state: current_term: 1 leader_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dd1868fd79f4536a1a70cc0c34ebef3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35579 } } }
I20260929 02:57:21.683943 29630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:21.683943 29629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:21.684727 29640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:21.689733 29640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:21.698695 29640 catalog_manager.cc:1347] Generated new cluster ID: 6a60390ef1ba4598884e904924ebf0cd
I20260929 02:57:21.698796 29640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:21.713464 29640 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:21.714726 29640 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:21.727905 29640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dd1868fd79f4536a1a70cc0c34ebef3: Generated new TSK 0
I20260929 02:57:21.728955 29640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:21.739818 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29551
W20260929 02:57:21.741214 29613 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.165:52605
W20260929 02:57:21.741950 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29551: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:21.742082 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29551
W20260929 02:57:21.755435 29536 connection.cc:535] client connection to 127.0.0.1:35579 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35579): Connection reset by peer (error 104)
2026-09-29T02:57:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1770 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:21 dist-test-slave-1dkt krb5kdc[29649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:21 dist-test-slave-1dkt krb5kdc[29649](info): set up 2 sockets
Sep 29 02:57:21 dist-test-slave-1dkt krb5kdc[29649](info): commencing operation
krb5kdc: starting...
W20260929 02:57:22.756181 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:22 dist-test-slave-1dkt krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:22.987159 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:23.308329 29662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:23.308593 29662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:23.308722 29662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:23.315737 29662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:23.315835 29662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:23.315951 29662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:23.316033 29662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:23.316141 29662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:23.325734 29662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:23.326930 29662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:23.328222 29662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:23.338984 29667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:23.339027 29668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:23.339048 29670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:23.339738 29662 server_base.cc:1034] running on GCE node
I20260929 02:57:23.340476 29662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:23.341753 29662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:23.342974 29662 hybrid_clock.cc:648] HybridClock initialized: now 1790650643342915 us; error 45 us; skew 500 ppm
I20260929 02:57:23.343315 29662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:23 dist-test-slave-1dkt krb5kdc[29649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:23.351221 29662 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:23.352643 29662 webserver.cc:460] Webserver started at http://127.0.0.1:40939/ using document root <none> and password file <none>
I20260929 02:57:23.353320 29662 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:23.353490 29662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:23.353770 29662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:23.356648 29662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "d855361d37f54d08bdb08dcd9befbb22"
format_stamp: "Formatted at 2026-09-29 02:57:23 on dist-test-slave-1dkt"
server_key: "1769350f16d67a1d0198efd06de1d309"
server_key_iv: "ce5eeff2a8e01db33f78638b53a84a04"
server_key_version: "encryptionkey@0"
I20260929 02:57:23.357393 29662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "d855361d37f54d08bdb08dcd9befbb22"
format_stamp: "Formatted at 2026-09-29 02:57:23 on dist-test-slave-1dkt"
server_key: "1769350f16d67a1d0198efd06de1d309"
server_key_iv: "ce5eeff2a8e01db33f78638b53a84a04"
server_key_version: "encryptionkey@0"
I20260929 02:57:23.362381 29662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:57:23.365700 29677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:23.367028 29662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:23.367166 29662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "d855361d37f54d08bdb08dcd9befbb22"
format_stamp: "Formatted at 2026-09-29 02:57:23 on dist-test-slave-1dkt"
server_key: "1769350f16d67a1d0198efd06de1d309"
server_key_iv: "ce5eeff2a8e01db33f78638b53a84a04"
server_key_version: "encryptionkey@0"
I20260929 02:57:23.367393 29662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:23.372876 29662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:23.376909 29662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:23.377153 29662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:23.394363 29662 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35089
I20260929 02:57:23.394397 29728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35089 every 8 connection(s)
I20260929 02:57:23.395777 29662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:23.398141 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29662
I20260929 02:57:23.398308 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:23.398732 8420 external_mini_cluster.cc:1428] Setting key 3d431f253cfc50372bb2c5fa47cbf923
I20260929 02:57:23.399719 29729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:23.406808 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:23.400399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:42685 (local address 127.0.0.1:35089)
0929 02:57:23.401167 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:23.401182 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:23.401229 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:57:23.403026 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:23.403054 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:23.403499 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:23.403958 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:23.404935 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:23.404944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:23.404965 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:23.404981 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:23.405063 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:23.405090 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:23.405311 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:23.405436 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 02:57:23.405621 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":318,"threads_started":1}
I20260929 02:57:23.410218 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:23.413883 29729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22: Bootstrap starting.
I20260929 02:57:23.417351 29729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:23.418543 29729 log.cc:824] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:23.419663 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:23.415268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:41357 (local address 127.0.0.1:35089)
0929 02:57:23.415445 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:23.415451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:23.415998 (+ 547us) server_negotiation.cc:407] Connection header received
0929 02:57:23.417552 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:23.417558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:23.417628 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:23.417760 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:23.418827 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:23.418833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:23.418837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:23.418859 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:23.418908 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:23.418928 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:23.419039 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:23.419307 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:57:23.419381 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:57:23.421793 29729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22: No bootstrap required, opened a new log
I20260929 02:57:23.430438 29729 raft_consensus.cc:348] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d855361d37f54d08bdb08dcd9befbb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260929 02:57:23.430924 29729 raft_consensus.cc:374] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:23.431030 29729 raft_consensus.cc:729] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d855361d37f54d08bdb08dcd9befbb22, State: Initialized, Role: FOLLOWER
I20260929 02:57:23.431710 29729 consensus_queue.cc:260] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d855361d37f54d08bdb08dcd9befbb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260929 02:57:23.432015 29729 raft_consensus.cc:388] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:23.432163 29729 raft_consensus.cc:482] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:23.432333 29729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:23.434588 29729 raft_consensus.cc:504] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d855361d37f54d08bdb08dcd9befbb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260929 02:57:23.435060 29729 leader_election.cc:302] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d855361d37f54d08bdb08dcd9befbb22; no voters:
I20260929 02:57:23.435621 29729 leader_election.cc:288] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:23.435729 29739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:23.437615 29729 sys_catalog.cc:564] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:23.440150 29739 raft_consensus.cc:686] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [term 1 LEADER]: Becoming Leader. State: Replica: d855361d37f54d08bdb08dcd9befbb22, State: Running, Role: LEADER
I20260929 02:57:23.440769 29739 consensus_queue.cc:237] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d855361d37f54d08bdb08dcd9befbb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } }
I20260929 02:57:23.443670 29740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d855361d37f54d08bdb08dcd9befbb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d855361d37f54d08bdb08dcd9befbb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } } }
I20260929 02:57:23.443725 29741 sys_catalog.cc:455] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d855361d37f54d08bdb08dcd9befbb22. Latest consensus state: current_term: 1 leader_uuid: "d855361d37f54d08bdb08dcd9befbb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d855361d37f54d08bdb08dcd9befbb22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35089 } } }
I20260929 02:57:23.444088 29740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:23.444195 29741 sys_catalog.cc:458] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:23.447790 29752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:23.452643 29752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:23.461171 29752 catalog_manager.cc:1347] Generated new cluster ID: 7cd5862f5fa04601882560b799567228
I20260929 02:57:23.461297 29752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:23.476227 29752 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:23.476598 29752 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:23.484551 29752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d855361d37f54d08bdb08dcd9befbb22: Generated new TSK 0
I20260929 02:57:23.485436 29752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:23.502063 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29662
W20260929 02:57:23.503083 29724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.165:42685
W20260929 02:57:23.503836 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29662: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:23.504016 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29662
2026-09-29T02:57:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1761 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:23 dist-test-slave-1dkt krb5kdc[29760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:23 dist-test-slave-1dkt krb5kdc[29760](info): set up 2 sockets
Sep 29 02:57:23 dist-test-slave-1dkt krb5kdc[29760](info): commencing operation
krb5kdc: starting...
W20260929 02:57:24.516052 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:24 dist-test-slave-1dkt krb5kdc[29760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:24.740568 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:25.049185 29773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:25.049449 29773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:25.049549 29773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:25.056607 29773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:25.056715 29773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:25.056778 29773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:25.056838 29773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:25.056897 29773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:25.066623 29773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:25.067821 29773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:25.069043 29773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:25.080138 29778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:25.080181 29781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:25.080204 29779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:25.080992 29773 server_base.cc:1034] running on GCE node
I20260929 02:57:25.081673 29773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:25.082948 29773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:25.084160 29773 hybrid_clock.cc:648] HybridClock initialized: now 1790650645084120 us; error 30 us; skew 500 ppm
I20260929 02:57:25.084486 29773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:25 dist-test-slave-1dkt krb5kdc[29760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:25.087747 29773 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:25.089210 29773 webserver.cc:460] Webserver started at http://127.0.0.1:38703/ using document root <none> and password file <none>
I20260929 02:57:25.089859 29773 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:25.089960 29773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:25.090235 29773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:25.093111 29773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb"
format_stamp: "Formatted at 2026-09-29 02:57:25 on dist-test-slave-1dkt"
server_key: "905d8ff030e051f9881c2ef8efe3fa5c"
server_key_iv: "e5cb1d81f8e27c53a80b925d2f3bba82"
server_key_version: "encryptionkey@0"
I20260929 02:57:25.093842 29773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb"
format_stamp: "Formatted at 2026-09-29 02:57:25 on dist-test-slave-1dkt"
server_key: "905d8ff030e051f9881c2ef8efe3fa5c"
server_key_iv: "e5cb1d81f8e27c53a80b925d2f3bba82"
server_key_version: "encryptionkey@0"
I20260929 02:57:25.098791 29773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260929 02:57:25.102123 29788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:25.103438 29773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:25.103600 29773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb"
format_stamp: "Formatted at 2026-09-29 02:57:25 on dist-test-slave-1dkt"
server_key: "905d8ff030e051f9881c2ef8efe3fa5c"
server_key_iv: "e5cb1d81f8e27c53a80b925d2f3bba82"
server_key_version: "encryptionkey@0"
I20260929 02:57:25.103780 29773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:25.117627 29773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:25.121598 29773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:25.121845 29773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:25.142585 29773 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38445
I20260929 02:57:25.142619 29839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38445 every 8 connection(s)
I20260929 02:57:25.143867 29773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:25.147414 29840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:25.150920 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29773
I20260929 02:57:25.151084 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:25.151484 8420 external_mini_cluster.cc:1428] Setting key ba77a5da1aca7bd3a23604d2c5c9d076
I20260929 02:57:25.159452 29840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb: Bootstrap starting.
I20260929 02:57:25.163059 29840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:25.164342 29840 log.cc:824] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:57:25 dist-test-slave-1dkt krb5kdc[29760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:57:25.167096 29840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb: No bootstrap required, opened a new log
I20260929 02:57:25.173715 29843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:25.153223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:44881 (local address 127.0.0.1:38445)
0929 02:57:25.153925 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:25.153935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:25.153975 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:57:25.155547 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:25.155569 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:25.156014 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:25.156336 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:25.158034 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:25.159327 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:25.161093 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:25.161754 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:25.161873 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:25.166604 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:25.166612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:25.166628 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:25.166664 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:25.169627 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:25.170451 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:25.170457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:25.170462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:25.170622 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:25.171054 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:25.171058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:25.171060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:25.171586 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:25.171932 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:25.172428 (+ 496us) server_negotiation.cc:299] Negotiation successful
0929 02:57:25.172744 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":304,"threads_started":1}
I20260929 02:57:25.174692 29840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38445 } }
I20260929 02:57:25.175112 29840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:25.175230 29840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1896cf3cd44c7993d338ce2ee0dcdb, State: Initialized, Role: FOLLOWER
I20260929 02:57:25.175853 29840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38445 } }
I20260929 02:57:25.176178 29840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:25.176327 29840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:25.176492 29840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:25.177508 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:25.179423 29840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38445 } }
I20260929 02:57:25.179997 29840 leader_election.cc:302] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a1896cf3cd44c7993d338ce2ee0dcdb; no voters:
I20260929 02:57:25.180619 29840 leader_election.cc:288] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:25.180737 29845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:25.183107 29845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [term 1 LEADER]: Becoming Leader. State: Replica: 4a1896cf3cd44c7993d338ce2ee0dcdb, State: Running, Role: LEADER
I20260929 02:57:25.184290 29840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:25.184381 29845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38445 } }
I20260929 02:57:25.195188 29848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38445 } } }
I20260929 02:57:25.195628 29848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:25.200197 29843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:25.183566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:33927 (local address 127.0.0.1:38445)
0929 02:57:25.183758 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:25.183763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:25.183781 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:57:25.183834 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:25.183838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:25.183934 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:25.184022 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:25.185902 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:25.187183 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:25.189163 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:25.189887 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:25.190039 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:25.193138 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:25.193144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:25.193148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:25.193194 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:25.197040 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:25.198052 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:25.198059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:25.198063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:25.198238 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:25.198694 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:25.198700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:25.198704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:25.199011 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:25.199292 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:25.199634 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 02:57:25.199806 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:57:25.201143 29849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a1896cf3cd44c7993d338ce2ee0dcdb. Latest consensus state: current_term: 1 leader_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1896cf3cd44c7993d338ce2ee0dcdb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38445 } } }
I20260929 02:57:25.201428 29849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb [sys.catalog]: This master's current role is: LEADER
W20260929 02:57:25.205996 29867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:57:25.206112 29867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:57:25.206260 29863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:25.208137 29863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:25.215461 29863 catalog_manager.cc:1347] Generated new cluster ID: 8ed212cd9b36418bbb32eef3c690f74e
I20260929 02:57:25.215596 29863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:25.234525 29863 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:25.235771 29863 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:25.241991 29863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a1896cf3cd44c7993d338ce2ee0dcdb: Generated new TSK 0
I20260929 02:57:25.242777 29863 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:25.254336 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29773
I20260929 02:57:25.255395 29835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:25.349380 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29773
W20260929 02:57:25.363055 29758 connection.cc:535] client connection to 127.0.0.1:38445 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38445): Connection reset by peer (error 104)
2026-09-29T02:57:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1846 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:25 dist-test-slave-1dkt krb5kdc[29873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:25 dist-test-slave-1dkt krb5kdc[29873](info): set up 2 sockets
Sep 29 02:57:25 dist-test-slave-1dkt krb5kdc[29873](info): commencing operation
krb5kdc: starting...
W20260929 02:57:26.372143 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:26 dist-test-slave-1dkt krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:26.602691 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:26.919131 29886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:26.919435 29886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:26.919569 29886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:26.926677 29886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:26.926793 29886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:26.926872 29886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:26.926951 29886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:26.927008 29886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:26.936610 29886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:26.937813 29886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:26.939052 29886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:26.950042 29894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:26.950081 29892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:26.950158 29891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:26.950819 29886 server_base.cc:1034] running on GCE node
I20260929 02:57:26.951609 29886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:26.953051 29886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:26.954267 29886 hybrid_clock.cc:648] HybridClock initialized: now 1790650646954221 us; error 43 us; skew 500 ppm
I20260929 02:57:26.954586 29886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:26 dist-test-slave-1dkt krb5kdc[29873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:26.957564 29886 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:26.958940 29886 webserver.cc:460] Webserver started at http://127.0.0.1:38085/ using document root <none> and password file <none>
I20260929 02:57:26.959558 29886 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:26.959666 29886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:26.959976 29886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:26.962842 29886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8"
format_stamp: "Formatted at 2026-09-29 02:57:26 on dist-test-slave-1dkt"
server_key: "5039a4a152dc77baf605cfb981eb6e00"
server_key_iv: "6385eccd9f5efd13e1e6dd04c119c5f1"
server_key_version: "encryptionkey@0"
I20260929 02:57:26.963546 29886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8"
format_stamp: "Formatted at 2026-09-29 02:57:26 on dist-test-slave-1dkt"
server_key: "5039a4a152dc77baf605cfb981eb6e00"
server_key_iv: "6385eccd9f5efd13e1e6dd04c119c5f1"
server_key_version: "encryptionkey@0"
I20260929 02:57:26.968408 29886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:57:26.971689 29901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:26.972975 29886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:26.973122 29886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8"
format_stamp: "Formatted at 2026-09-29 02:57:26 on dist-test-slave-1dkt"
server_key: "5039a4a152dc77baf605cfb981eb6e00"
server_key_iv: "6385eccd9f5efd13e1e6dd04c119c5f1"
server_key_version: "encryptionkey@0"
I20260929 02:57:26.973309 29886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:26.983510 29886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:26.987156 29886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:26.987428 29886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:27.003965 29886 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46835
I20260929 02:57:27.003988 29952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46835 every 8 connection(s)
I20260929 02:57:27.005282 29886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:27.008790 29953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:27.013012 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29886
I20260929 02:57:27.013170 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:27.013556 8420 external_mini_cluster.cc:1428] Setting key 7a138e8b78f65d90dc2fe593abc1442a
I20260929 02:57:27.019378 29953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8: Bootstrap starting.
I20260929 02:57:27.022814 29953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:27.024008 29953 log.cc:824] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:27.026733 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:27.015169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:55221 (local address 127.0.0.1:46835)
0929 02:57:27.015854 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:27.015864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:27.015932 (+ 68us) server_negotiation.cc:407] Connection header received
0929 02:57:27.017934 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:27.017954 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:27.018332 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:27.018623 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:27.019187 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:27.020569 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:27.022437 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:27.023106 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:27.023244 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:27.024457 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:27.024464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:27.024481 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:27.025191 (+ 710us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:27.025267 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:27.025296 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:27.025434 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:27.025670 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 02:57:27.025831 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":302,"threads_started":1}
I20260929 02:57:27.026932 29953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8: No bootstrap required, opened a new log
I20260929 02:57:27.030184 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:27.036054 29953 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260929 02:57:27.036441 29953 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:27.036566 29953 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcf3a6295db4d2d80bbfa53fba6b4a8, State: Initialized, Role: FOLLOWER
I20260929 02:57:27.037180 29953 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260929 02:57:27.037467 29953 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:27.037612 29953 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:27.037765 29953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:27.040355 29953 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260929 02:57:27.040921 29953 leader_election.cc:302] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebcf3a6295db4d2d80bbfa53fba6b4a8; no voters:
I20260929 02:57:27.041307 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:27.034976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:40861 (local address 127.0.0.1:46835)
0929 02:57:27.035144 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:27.035152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:27.035586 (+ 434us) server_negotiation.cc:407] Connection header received
0929 02:57:27.035723 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:27.035728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:27.035819 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:27.035965 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:27.036414 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:27.037480 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:27.038847 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:27.039515 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:27.039630 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:27.040586 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:27.040591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:27.040595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:27.040615 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:27.040661 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:27.040687 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:27.040804 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:27.040953 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 02:57:27.041004 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:57:27.041610 29953 leader_election.cc:288] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:27.041746 29963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:27.043218 29963 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [term 1 LEADER]: Becoming Leader. State: Replica: ebcf3a6295db4d2d80bbfa53fba6b4a8, State: Running, Role: LEADER
I20260929 02:57:27.043498 29953 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:27.043740 29963 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260929 02:57:27.046433 29964 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } } }
I20260929 02:57:27.046423 29965 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcf3a6295db4d2d80bbfa53fba6b4a8. Latest consensus state: current_term: 1 leader_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcf3a6295db4d2d80bbfa53fba6b4a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } } }
I20260929 02:57:27.046849 29964 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:27.046849 29965 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:27.048420 29969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:27.053339 29969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:27.062146 29969 catalog_manager.cc:1347] Generated new cluster ID: 2d505c7ab29347e894d484c68a8ea213
I20260929 02:57:27.062259 29969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:27.082988 29969 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:27.084286 29969 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:27.097569 29969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebcf3a6295db4d2d80bbfa53fba6b4a8: Generated new TSK 0
I20260929 02:57:27.098335 29969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:27.109182 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29886
W20260929 02:57:27.110699 29948 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.165:55221
W20260929 02:57:27.111469 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:27.111642 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29886
W20260929 02:57:27.127627 29871 connection.cc:535] client connection to 127.0.0.1:46835 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46835): Connection reset by peer (error 104)
2026-09-29T02:57:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1764 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:27 dist-test-slave-1dkt krb5kdc[29984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:27 dist-test-slave-1dkt krb5kdc[29984](info): set up 2 sockets
Sep 29 02:57:27 dist-test-slave-1dkt krb5kdc[29984](info): commencing operation
krb5kdc: starting...
W20260929 02:57:28.128233 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:28 dist-test-slave-1dkt krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:57:28.367357 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:28.679013 29997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:28.679287 29997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:28.679378 29997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:28.686518 29997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:28.686619 29997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:28.686671 29997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:28.686729 29997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:28.686770 29997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:28.696286 29997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35385
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:28.697481 29997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:28.698710 29997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:28.709446 30002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:28.709497 30005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:28.709419 30003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:28.710233 29997 server_base.cc:1034] running on GCE node
I20260929 02:57:28.710971 29997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:28.712342 29997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:28.713536 29997 hybrid_clock.cc:648] HybridClock initialized: now 1790650648713487 us; error 44 us; skew 500 ppm
I20260929 02:57:28.713850 29997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:28 dist-test-slave-1dkt krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:28.720499 29997 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:57:28.721930 29997 webserver.cc:460] Webserver started at http://127.0.0.1:36695/ using document root <none> and password file <none>
I20260929 02:57:28.722581 29997 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:28.722699 29997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:28.722983 29997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:28.725937 29997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "69937e31100642ed9c8da5710b9bafe6"
format_stamp: "Formatted at 2026-09-29 02:57:28 on dist-test-slave-1dkt"
server_key: "625495a9184c508a60ca78c161f524f8"
server_key_iv: "45c63ee751edeba119e2e7731942c19c"
server_key_version: "encryptionkey@0"
I20260929 02:57:28.726677 29997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "69937e31100642ed9c8da5710b9bafe6"
format_stamp: "Formatted at 2026-09-29 02:57:28 on dist-test-slave-1dkt"
server_key: "625495a9184c508a60ca78c161f524f8"
server_key_iv: "45c63ee751edeba119e2e7731942c19c"
server_key_version: "encryptionkey@0"
I20260929 02:57:28.731691 29997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:57:28.734936 30012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:28.736172 29997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:57:28.736342 29997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "69937e31100642ed9c8da5710b9bafe6"
format_stamp: "Formatted at 2026-09-29 02:57:28 on dist-test-slave-1dkt"
server_key: "625495a9184c508a60ca78c161f524f8"
server_key_iv: "45c63ee751edeba119e2e7731942c19c"
server_key_version: "encryptionkey@0"
I20260929 02:57:28.736517 29997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:28.741835 29997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:28.746143 29997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:28.746445 29997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:28.763311 29997 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35385
I20260929 02:57:28.763365 30063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35385 every 8 connection(s)
I20260929 02:57:28.764827 29997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:28.768028 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 29997
I20260929 02:57:28.768194 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:28.768530 30064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:28.768638 8420 external_mini_cluster.cc:1428] Setting key 487ebf8332667aa04ae052eb4bdf0ed2
I20260929 02:57:28.776222 30067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:28.770263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:49957 (local address 127.0.0.1:35385)
0929 02:57:28.770976 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:28.770987 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:28.771029 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:57:28.772979 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:28.773000 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:28.773349 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:28.773661 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:28.774679 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:28.774685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:28.774697 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:28.774710 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:28.774771 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:28.774794 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:28.774963 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:28.775054 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 02:57:28.775231 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":298,"threads_started":1}
I20260929 02:57:28.779716 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:28.781497 30064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6: Bootstrap starting.
I20260929 02:57:28.784685 30064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:28.785812 30064 log.cc:824] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:28.786046 30067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:28.783771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.165:38405 (local address 127.0.0.1:35385)
0929 02:57:28.783962 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:28.783967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:28.784354 (+ 387us) server_negotiation.cc:407] Connection header received
0929 02:57:28.784489 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:28.784494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:28.784558 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:28.784664 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:28.785457 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:28.785462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:28.785470 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:28.785489 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:28.785534 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:28.785554 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:28.785672 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:28.785720 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 02:57:28.785773 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:57:28.788733 30064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6: No bootstrap required, opened a new log
I20260929 02:57:28.795811 30064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69937e31100642ed9c8da5710b9bafe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35385 } }
I20260929 02:57:28.796197 30064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:28.796274 30064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69937e31100642ed9c8da5710b9bafe6, State: Initialized, Role: FOLLOWER
I20260929 02:57:28.796820 30064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69937e31100642ed9c8da5710b9bafe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35385 } }
I20260929 02:57:28.797044 30064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:28.797186 30064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:28.797361 30064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:28.799502 30064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69937e31100642ed9c8da5710b9bafe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35385 } }
I20260929 02:57:28.800012 30064 leader_election.cc:302] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69937e31100642ed9c8da5710b9bafe6; no voters:
I20260929 02:57:28.800563 30064 leader_election.cc:288] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:28.800702 30074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:28.801863 30074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [term 1 LEADER]: Becoming Leader. State: Replica: 69937e31100642ed9c8da5710b9bafe6, State: Running, Role: LEADER
I20260929 02:57:28.802419 30074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69937e31100642ed9c8da5710b9bafe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35385 } }
I20260929 02:57:28.802968 30064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:28.804931 30075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69937e31100642ed9c8da5710b9bafe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69937e31100642ed9c8da5710b9bafe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35385 } } }
I20260929 02:57:28.805338 30075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:28.805269 30076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69937e31100642ed9c8da5710b9bafe6. Latest consensus state: current_term: 1 leader_uuid: "69937e31100642ed9c8da5710b9bafe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69937e31100642ed9c8da5710b9bafe6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35385 } } }
I20260929 02:57:28.805516 30076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:28.806931 30082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:28.811612 30082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:28.819370 30082 catalog_manager.cc:1347] Generated new cluster ID: cb6540b00db84a9fb9a16909181defc8
I20260929 02:57:28.819480 30082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:28.832954 30082 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:28.833492 30082 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:28.843564 30082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69937e31100642ed9c8da5710b9bafe6: Generated new TSK 0
I20260929 02:57:28.844605 30082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:28.854079 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29997
W20260929 02:57:28.855518 30059 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.165:49957
W20260929 02:57:28.856176 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 29997: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:28.856330 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 29997
2026-09-29T02:57:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1741 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:28 dist-test-slave-1dkt krb5kdc[30095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:28 dist-test-slave-1dkt krb5kdc[30095](info): set up 2 sockets
Sep 29 02:57:28 dist-test-slave-1dkt krb5kdc[30095](info): commencing operation
krb5kdc: starting...
W20260929 02:57:29.856043 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:30 dist-test-slave-1dkt krb5kdc[30095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:30.085152 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:34293
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:39403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:34293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:30.397104 30108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:30.397377 30108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:30.397509 30108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:30.404553 30108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:30.404650 30108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:30.404726 30108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:30.404806 30108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:30.404865 30108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:30.414300 30108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:39403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:34293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:34293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:30.415454 30108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:30.416713 30108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:30.427546 30114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:30.427591 30113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:30.427605 30116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:30.428351 30108 server_base.cc:1034] running on GCE node
I20260929 02:57:30.429005 30108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:30.430235 30108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:30.431416 30108 hybrid_clock.cc:648] HybridClock initialized: now 1790650650431368 us; error 43 us; skew 500 ppm
I20260929 02:57:30.431695 30108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:30 dist-test-slave-1dkt krb5kdc[30095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650650, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:30.434577 30108 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:30.435933 30108 webserver.cc:460] Webserver started at http://127.8.57.62:44323/ using document root <none> and password file <none>
I20260929 02:57:30.436538 30108 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:30.436630 30108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:30.436868 30108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:30.439563 30108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "c37a8f4020494437a82cf5799c647249"
format_stamp: "Formatted at 2026-09-29 02:57:30 on dist-test-slave-1dkt"
server_key: "bbc82493a74cfc608ec99c2a40aa7a50"
server_key_iv: "4d872a48a50d8f3a3d26996b19f8c79c"
server_key_version: "encryptionkey@0"
I20260929 02:57:30.440274 30108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "c37a8f4020494437a82cf5799c647249"
format_stamp: "Formatted at 2026-09-29 02:57:30 on dist-test-slave-1dkt"
server_key: "bbc82493a74cfc608ec99c2a40aa7a50"
server_key_iv: "4d872a48a50d8f3a3d26996b19f8c79c"
server_key_version: "encryptionkey@0"
I20260929 02:57:30.444888 30108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:57:30.448067 30123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:30.449339 30108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:57:30.449491 30108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "c37a8f4020494437a82cf5799c647249"
format_stamp: "Formatted at 2026-09-29 02:57:30 on dist-test-slave-1dkt"
server_key: "bbc82493a74cfc608ec99c2a40aa7a50"
server_key_iv: "4d872a48a50d8f3a3d26996b19f8c79c"
server_key_version: "encryptionkey@0"
I20260929 02:57:30.449658 30108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:30.462116 30108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:30.466009 30108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:30.466248 30108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:30.482509 30108 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:34293
I20260929 02:57:30.482540 30174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:34293 every 8 connection(s)
I20260929 02:57:30.483719 30108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:30.485270 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30108
I20260929 02:57:30.485419 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:30.485816 8420 external_mini_cluster.cc:1428] Setting key 91e20eb98d66d64aa4e3b6006a80507a
I20260929 02:57:30.487550 30175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:57:30 dist-test-slave-1dkt krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:30.499928 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249: Bootstrap starting.
I20260929 02:57:30.503888 30175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:30.505051 30175 log.cc:824] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:30.508320 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249: No bootstrap required, opened a new log
I20260929 02:57:30.508386 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:30.487312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42231 (local address 127.8.57.62:34293)
0929 02:57:30.488049 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:30.488063 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:30.488101 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:57:30.489692 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:30.489711 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:30.490069 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:30.490379 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:30.491991 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:30.493240 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:30.494902 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:30.495530 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:30.495651 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:30.500229 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:30.500241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:30.500258 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:30.500305 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:30.503961 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:30.504669 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:30.504676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:30.504684 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:30.504893 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:30.505260 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:30.505265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:30.505268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:30.505912 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:30.506177 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:30.506707 (+ 530us) server_negotiation.cc:299] Negotiation successful
0929 02:57:30.507037 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":303,"threads_started":1}
I20260929 02:57:30.511986 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:30.517527 30175 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37a8f4020494437a82cf5799c647249" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34293 } }
I20260929 02:57:30.518009 30175 raft_consensus.cc:374] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:30.518105 30175 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37a8f4020494437a82cf5799c647249, State: Initialized, Role: FOLLOWER
I20260929 02:57:30.518636 30175 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37a8f4020494437a82cf5799c647249" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34293 } }
I20260929 02:57:30.518851 30175 raft_consensus.cc:388] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:30.518977 30175 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:30.519120 30175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:30.521910 30175 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37a8f4020494437a82cf5799c647249" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34293 } }
I20260929 02:57:30.522522 30175 leader_election.cc:302] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c37a8f4020494437a82cf5799c647249; no voters:
I20260929 02:57:30.523234 30175 leader_election.cc:288] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:30.523373 30185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:30.524559 30185 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [term 1 LEADER]: Becoming Leader. State: Replica: c37a8f4020494437a82cf5799c647249, State: Running, Role: LEADER
I20260929 02:57:30.525188 30185 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37a8f4020494437a82cf5799c647249" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34293 } }
I20260929 02:57:30.525499 30175 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:30.528076 30187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37a8f4020494437a82cf5799c647249. Latest consensus state: current_term: 1 leader_uuid: "c37a8f4020494437a82cf5799c647249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37a8f4020494437a82cf5799c647249" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34293 } } }
I20260929 02:57:30.528510 30187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:30.530611 30186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37a8f4020494437a82cf5799c647249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37a8f4020494437a82cf5799c647249" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 34293 } } }
I20260929 02:57:30.530943 30186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:30.531713 30191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:30.533806 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:30.517020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53515 (local address 127.8.57.62:34293)
0929 02:57:30.517190 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:30.517196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:30.517562 (+ 366us) server_negotiation.cc:407] Connection header received
0929 02:57:30.517687 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:30.517691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:30.517765 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:30.517872 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:30.519296 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:30.520522 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:30.521709 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:30.522294 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:30.522426 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:30.525438 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:30.525444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:30.525448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:30.525491 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:30.529463 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:30.531181 (+ 1718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:30.531187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:30.531195 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:30.531424 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:30.532074 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:30.532080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:30.532083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:30.532505 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:30.532835 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:30.533224 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 02:57:30.533424 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:57:30.538607 30191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:30.547564 30191 catalog_manager.cc:1347] Generated new cluster ID: 97bbdc7a9b1a4d2f823ef4a276490701
I20260929 02:57:30.547680 30191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:30.559254 30191 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:30.561017 30191 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:30.573489 30191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37a8f4020494437a82cf5799c647249: Generated new TSK 0
I20260929 02:57:30.574498 30191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:30.587710 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30108
I20260929 02:57:30.588738 30170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:30.676720 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30108
W20260929 02:57:30.689913 30093 connection.cc:535] client connection to 127.8.57.62:34293 recv error: Network error: failed to read from TLS socket (remote: 127.8.57.62:34293): Connection reset by peer (error 104)
2026-09-29T02:57:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1820 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:30 dist-test-slave-1dkt krb5kdc[30207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:30 dist-test-slave-1dkt krb5kdc[30207](info): set up 2 sockets
Sep 29 02:57:30 dist-test-slave-1dkt krb5kdc[30207](info): commencing operation
krb5kdc: starting...
W20260929 02:57:31.684252 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:31 dist-test-slave-1dkt krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:31.910997 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:43243
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:45273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:43243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:32.222424 30220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:32.222703 30220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:32.222836 30220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:32.229888 30220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:32.229992 30220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:32.230069 30220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:32.230146 30220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:32.230201 30220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:32.239626 30220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:45273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:43243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:43243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:32.240864 30220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:32.242082 30220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:32.253088 30225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:32.253134 30228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:32.253167 30226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:32.254033 30220 server_base.cc:1034] running on GCE node
I20260929 02:57:32.254773 30220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:32.256083 30220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:32.257269 30220 hybrid_clock.cc:648] HybridClock initialized: now 1790650652257228 us; error 32 us; skew 500 ppm
I20260929 02:57:32.257606 30220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:32 dist-test-slave-1dkt krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:32.260581 30220 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:32.261963 30220 webserver.cc:460] Webserver started at http://127.8.57.62:42607/ using document root <none> and password file <none>
I20260929 02:57:32.262578 30220 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:32.262676 30220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:32.262933 30220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:32.265870 30220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "0ab83819630046e19d9d30b95f1fc1f6"
format_stamp: "Formatted at 2026-09-29 02:57:32 on dist-test-slave-1dkt"
server_key: "e415c8efb1b1bb6642fb1d8f02cae523"
server_key_iv: "9936a9ff29fd78156e6292f1bad61486"
server_key_version: "encryptionkey@0"
I20260929 02:57:32.266642 30220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "0ab83819630046e19d9d30b95f1fc1f6"
format_stamp: "Formatted at 2026-09-29 02:57:32 on dist-test-slave-1dkt"
server_key: "e415c8efb1b1bb6642fb1d8f02cae523"
server_key_iv: "9936a9ff29fd78156e6292f1bad61486"
server_key_version: "encryptionkey@0"
I20260929 02:57:32.271817 30220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:57:32.275445 30235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:32.276924 30220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:32.277077 30220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "0ab83819630046e19d9d30b95f1fc1f6"
format_stamp: "Formatted at 2026-09-29 02:57:32 on dist-test-slave-1dkt"
server_key: "e415c8efb1b1bb6642fb1d8f02cae523"
server_key_iv: "9936a9ff29fd78156e6292f1bad61486"
server_key_version: "encryptionkey@0"
I20260929 02:57:32.277246 30220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:32.302716 30220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:32.306524 30220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:32.306756 30220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:32.323585 30220 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:43243
I20260929 02:57:32.323613 30286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:43243 every 8 connection(s)
I20260929 02:57:32.324915 30220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:32.328524 30287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:32.331027 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30220
I20260929 02:57:32.331179 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:32.331581 8420 external_mini_cluster.cc:1428] Setting key ce3fe2c59b9b914c68d137a528e0cf09
I20260929 02:57:32.339316 30287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6: Bootstrap starting.
I20260929 02:57:32.342561 30287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:32.343624 30287 log.cc:824] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:32.344058 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:32.333251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39923 (local address 127.8.57.62:43243)
0929 02:57:32.333970 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:32.333981 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:32.334020 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:57:32.335607 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:32.335628 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:32.336054 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:32.336380 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:32.336927 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:32.338217 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:32.339949 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:32.340578 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:32.340714 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:32.341908 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:32.341916 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:32.341927 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:32.342425 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:32.342502 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:32.342525 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:32.342695 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:32.342898 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 02:57:32.343069 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":316,"threads_started":1}
I20260929 02:57:32.346625 30287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6: No bootstrap required, opened a new log
I20260929 02:57:32.347849 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:32.356212 30287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab83819630046e19d9d30b95f1fc1f6" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43243 } }
I20260929 02:57:32.356652 30287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:32.356752 30287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab83819630046e19d9d30b95f1fc1f6, State: Initialized, Role: FOLLOWER
I20260929 02:57:32.357314 30287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab83819630046e19d9d30b95f1fc1f6" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43243 } }
I20260929 02:57:32.357589 30287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:32.357721 30287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:32.357882 30287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:32.359079 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:32.352713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59715 (local address 127.8.57.62:43243)
0929 02:57:32.352898 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:32.352903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:32.353193 (+ 290us) server_negotiation.cc:407] Connection header received
0929 02:57:32.353396 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:32.353400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:32.353462 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:32.353553 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:32.354047 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:32.354824 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:32.356565 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:32.357081 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:32.357188 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:32.358316 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:32.358321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:32.358324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:32.358339 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:32.358398 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:32.358413 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:32.358503 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:32.358712 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 02:57:32.358760 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:57:32.360512 30287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab83819630046e19d9d30b95f1fc1f6" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43243 } }
I20260929 02:57:32.360955 30287 leader_election.cc:302] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ab83819630046e19d9d30b95f1fc1f6; no voters:
I20260929 02:57:32.361447 30287 leader_election.cc:288] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:32.361545 30297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:32.362112 30297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 0ab83819630046e19d9d30b95f1fc1f6, State: Running, Role: LEADER
I20260929 02:57:32.362697 30297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab83819630046e19d9d30b95f1fc1f6" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43243 } }
I20260929 02:57:32.364331 30297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ab83819630046e19d9d30b95f1fc1f6. Latest consensus state: current_term: 1 leader_uuid: "0ab83819630046e19d9d30b95f1fc1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab83819630046e19d9d30b95f1fc1f6" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43243 } } }
I20260929 02:57:32.364552 30297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:32.365744 30287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:32.365585 30299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ab83819630046e19d9d30b95f1fc1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab83819630046e19d9d30b95f1fc1f6" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 43243 } } }
I20260929 02:57:32.365942 30301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:32.366052 30299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:32.371472 30301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:32.379781 30301 catalog_manager.cc:1347] Generated new cluster ID: 42448876de4f48c7b804fa4103fa6dbf
I20260929 02:57:32.379899 30301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:32.408334 30301 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:32.409576 30301 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:32.415784 30301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ab83819630046e19d9d30b95f1fc1f6: Generated new TSK 0
I20260929 02:57:32.416750 30301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:32.432015 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30220
W20260929 02:57:32.433034 30282 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39923
W20260929 02:57:32.433678 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30220: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:32.433794 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30220
W20260929 02:57:32.446560 30205 connection.cc:535] client connection to 127.8.57.62:43243 recv error: Network error: failed to read from TLS socket (remote: 127.8.57.62:43243): Connection reset by peer (error 104)
2026-09-29T02:57:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1756 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:32 dist-test-slave-1dkt krb5kdc[30318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:32 dist-test-slave-1dkt krb5kdc[30318](info): set up 2 sockets
Sep 29 02:57:32 dist-test-slave-1dkt krb5kdc[30318](info): commencing operation
krb5kdc: starting...
W20260929 02:57:33.444142 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:33 dist-test-slave-1dkt krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:33.675180 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:32783
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:32783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:33.987315 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:33.987608 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:33.987705 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:33.994874 30331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:33.994984 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:33.995036 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:33.995095 30331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:33.995138 30331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:34.004906 30331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:32783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:32783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:34.006106 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:34.007365 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:34.018711 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:34.018749 30339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:34.018777 30337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:34.019520 30331 server_base.cc:1034] running on GCE node
I20260929 02:57:34.020351 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:34.021770 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:34.022993 30331 hybrid_clock.cc:648] HybridClock initialized: now 1790650654022939 us; error 44 us; skew 500 ppm
I20260929 02:57:34.023310 30331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:34 dist-test-slave-1dkt krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650654, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:34.030231 30331 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:34.031678 30331 webserver.cc:460] Webserver started at http://127.8.57.62:42581/ using document root <none> and password file <none>
I20260929 02:57:34.032354 30331 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:34.032456 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:34.032703 30331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:34.035598 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "7e4f546e6d4647bd92432b9bcc14e71d"
format_stamp: "Formatted at 2026-09-29 02:57:34 on dist-test-slave-1dkt"
server_key: "13a6d3008ff509555daf88d37581065c"
server_key_iv: "4b05b1c2d96a5bd029b680e760e1bdd7"
server_key_version: "encryptionkey@0"
I20260929 02:57:34.036329 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "7e4f546e6d4647bd92432b9bcc14e71d"
format_stamp: "Formatted at 2026-09-29 02:57:34 on dist-test-slave-1dkt"
server_key: "13a6d3008ff509555daf88d37581065c"
server_key_iv: "4b05b1c2d96a5bd029b680e760e1bdd7"
server_key_version: "encryptionkey@0"
I20260929 02:57:34.041240 30331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 02:57:34.044523 30346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:34.046012 30331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:57:34.046216 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "7e4f546e6d4647bd92432b9bcc14e71d"
format_stamp: "Formatted at 2026-09-29 02:57:34 on dist-test-slave-1dkt"
server_key: "13a6d3008ff509555daf88d37581065c"
server_key_iv: "4b05b1c2d96a5bd029b680e760e1bdd7"
server_key_version: "encryptionkey@0"
I20260929 02:57:34.046402 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:34.052016 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:34.056008 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:34.056245 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:34.072760 30331 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:32783
I20260929 02:57:34.072798 30397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:32783 every 8 connection(s)
I20260929 02:57:34.074079 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:34.075464 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30331
I20260929 02:57:34.075616 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:34.076077 8420 external_mini_cluster.cc:1428] Setting key 398cf92aa5df237f7785a2f95fab2c76
I20260929 02:57:34.078142 30398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:34.083530 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:34.077736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37481 (local address 127.8.57.62:32783)
0929 02:57:34.078495 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:34.078506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:34.078545 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:57:34.080473 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:34.080493 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:34.080822 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:34.081144 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:34.082116 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:34.082122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:34.082133 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:34.082146 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:34.082218 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:34.082237 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:34.082397 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:34.082483 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 02:57:34.082637 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":322,"threads_started":1}
I20260929 02:57:34.086774 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:34.089461 30398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d: Bootstrap starting.
I20260929 02:57:34.092645 30398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:34.092861 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:34.090448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52171 (local address 127.8.57.62:32783)
0929 02:57:34.090568 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:34.090572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:34.091031 (+ 459us) server_negotiation.cc:407] Connection header received
0929 02:57:34.091181 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:34.091185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:34.091225 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:34.091306 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:34.092194 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:34.092200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:34.092205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:34.092221 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:34.092257 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:34.092271 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:34.092358 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:34.092495 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 02:57:34.092551 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 02:57:34.093838 30398 log.cc:824] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:34.096220 30398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d: No bootstrap required, opened a new log
I20260929 02:57:34.103111 30398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 32783 } }
I20260929 02:57:34.103440 30398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:34.103530 30398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e4f546e6d4647bd92432b9bcc14e71d, State: Initialized, Role: FOLLOWER
I20260929 02:57:34.104079 30398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 32783 } }
I20260929 02:57:34.104270 30398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:34.104410 30398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:34.104552 30398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:34.106516 30398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 32783 } }
I20260929 02:57:34.106945 30398 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e4f546e6d4647bd92432b9bcc14e71d; no voters:
I20260929 02:57:34.107448 30398 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:34.107546 30408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:34.108188 30408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [term 1 LEADER]: Becoming Leader. State: Replica: 7e4f546e6d4647bd92432b9bcc14e71d, State: Running, Role: LEADER
I20260929 02:57:34.108701 30408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 32783 } }
I20260929 02:57:34.109350 30398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:34.111310 30410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4f546e6d4647bd92432b9bcc14e71d. Latest consensus state: current_term: 1 leader_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 32783 } } }
I20260929 02:57:34.111325 30409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4f546e6d4647bd92432b9bcc14e71d" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 32783 } } }
I20260929 02:57:34.111780 30409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:34.111780 30410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:34.112654 30414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:34.117411 30414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:34.126657 30414 catalog_manager.cc:1347] Generated new cluster ID: 9e2e29e9e3414051b5ff84a8e8701b99
I20260929 02:57:34.126765 30414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:34.140870 30414 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:34.141270 30414 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:34.149166 30414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e4f546e6d4647bd92432b9bcc14e71d: Generated new TSK 0
I20260929 02:57:34.150075 30414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:34.160842 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30331
W20260929 02:57:34.161978 30393 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37481
W20260929 02:57:34.162632 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30331: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:34.162757 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30331
2026-09-29T02:57:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1730 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:34 dist-test-slave-1dkt krb5kdc[30429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:34 dist-test-slave-1dkt krb5kdc[30429](info): set up 2 sockets
Sep 29 02:57:34 dist-test-slave-1dkt krb5kdc[30429](info): commencing operation
krb5kdc: starting...
W20260929 02:57:35.184108 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:35 dist-test-slave-1dkt krb5kdc[30429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:35.406181 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42811
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:42811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:35.712826 30442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:35.713124 30442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:35.713260 30442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:35.720510 30442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:35.720630 30442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:35.720705 30442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:35.720794 30442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:35.720852 30442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:35.730662 30442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:42811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:35.731897 30442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:35.733162 30442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:35.744060 30447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:35.744014 30448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:35.744220 30450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:35.744922 30442 server_base.cc:1034] running on GCE node
I20260929 02:57:35.745617 30442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:35.746887 30442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:35.748070 30442 hybrid_clock.cc:648] HybridClock initialized: now 1790650655748018 us; error 45 us; skew 500 ppm
I20260929 02:57:35.748400 30442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:35 dist-test-slave-1dkt krb5kdc[30429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650655, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:35.751233 30442 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:35.752604 30442 webserver.cc:460] Webserver started at http://127.8.57.62:35199/ using document root <none> and password file <none>
I20260929 02:57:35.753250 30442 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:35.753363 30442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:35.753636 30442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:35.756464 30442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "da9f8e9b46da4445a89bdeda6c565498"
format_stamp: "Formatted at 2026-09-29 02:57:35 on dist-test-slave-1dkt"
server_key: "eba4910706fe12d2b1275a4940549e84"
server_key_iv: "02a3cc505de40f27c279ac7059407f1e"
server_key_version: "encryptionkey@0"
I20260929 02:57:35.757164 30442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "da9f8e9b46da4445a89bdeda6c565498"
format_stamp: "Formatted at 2026-09-29 02:57:35 on dist-test-slave-1dkt"
server_key: "eba4910706fe12d2b1275a4940549e84"
server_key_iv: "02a3cc505de40f27c279ac7059407f1e"
server_key_version: "encryptionkey@0"
I20260929 02:57:35.761999 30442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:57:35.765228 30457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:35.766393 30442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:57:35.766544 30442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "da9f8e9b46da4445a89bdeda6c565498"
format_stamp: "Formatted at 2026-09-29 02:57:35 on dist-test-slave-1dkt"
server_key: "eba4910706fe12d2b1275a4940549e84"
server_key_iv: "02a3cc505de40f27c279ac7059407f1e"
server_key_version: "encryptionkey@0"
I20260929 02:57:35.766717 30442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:35.780670 30442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:35.784336 30442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:35.784576 30442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:35.800675 30442 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:42811
I20260929 02:57:35.800701 30508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:42811 every 8 connection(s)
I20260929 02:57:35.801890 30442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:35.805282 30509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:35.805384 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30442
I20260929 02:57:35.805545 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:35.806088 8420 external_mini_cluster.cc:1428] Setting key c18ebb2d2cd438f89b0d70636a7eb4ae
I20260929 02:57:35.818534 30509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498: Bootstrap starting.
Sep 29 02:57:35 dist-test-slave-1dkt krb5kdc[30429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:35.821851 30509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:35.822870 30509 log.cc:824] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:35.825655 30509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498: No bootstrap required, opened a new log
I20260929 02:57:35.827024 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:35.807673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40875 (local address 127.8.57.62:42811)
0929 02:57:35.808467 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:35.808480 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:35.808520 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:57:35.809971 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:35.809989 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:35.810355 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:35.810665 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:35.812328 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:35.813532 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:35.815214 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:35.815863 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:35.816020 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:35.820598 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:35.820605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:35.820618 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:35.820649 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:35.823511 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:35.824199 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:35.824205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:35.824211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:35.824360 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:35.824710 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:35.824713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:35.824715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:35.825150 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:35.825360 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:35.825772 (+ 412us) server_negotiation.cc:299] Negotiation successful
0929 02:57:35.826090 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":386,"threads_started":1}
I20260929 02:57:35.830443 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:35.834985 30509 raft_consensus.cc:348] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9f8e9b46da4445a89bdeda6c565498" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42811 } }
I20260929 02:57:35.835359 30509 raft_consensus.cc:374] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:35.835456 30509 raft_consensus.cc:729] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9f8e9b46da4445a89bdeda6c565498, State: Initialized, Role: FOLLOWER
I20260929 02:57:35.836035 30509 consensus_queue.cc:260] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9f8e9b46da4445a89bdeda6c565498" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42811 } }
I20260929 02:57:35.836259 30509 raft_consensus.cc:388] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:35.836382 30509 raft_consensus.cc:482] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:35.836532 30509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:35.838570 30509 raft_consensus.cc:504] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9f8e9b46da4445a89bdeda6c565498" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42811 } }
I20260929 02:57:35.838989 30509 leader_election.cc:302] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da9f8e9b46da4445a89bdeda6c565498; no voters:
I20260929 02:57:35.839537 30509 leader_election.cc:288] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:35.839645 30519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:35.841672 30509 sys_catalog.cc:564] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:35.843814 30519 raft_consensus.cc:686] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [term 1 LEADER]: Becoming Leader. State: Replica: da9f8e9b46da4445a89bdeda6c565498, State: Running, Role: LEADER
I20260929 02:57:35.844465 30519 consensus_queue.cc:237] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9f8e9b46da4445a89bdeda6c565498" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42811 } }
I20260929 02:57:35.847434 30520 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da9f8e9b46da4445a89bdeda6c565498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9f8e9b46da4445a89bdeda6c565498" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42811 } } }
I20260929 02:57:35.847410 30521 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da9f8e9b46da4445a89bdeda6c565498. Latest consensus state: current_term: 1 leader_uuid: "da9f8e9b46da4445a89bdeda6c565498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9f8e9b46da4445a89bdeda6c565498" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 42811 } } }
I20260929 02:57:35.847946 30520 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:35.847983 30521 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:35.849457 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:35.835297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58127 (local address 127.8.57.62:42811)
0929 02:57:35.835433 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:35.835437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:35.836044 (+ 607us) server_negotiation.cc:407] Connection header received
0929 02:57:35.836188 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:35.836193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:35.836262 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:35.836342 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:35.837567 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:35.838236 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:35.839472 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:35.840070 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:35.840199 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:35.842613 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:35.842618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:35.842621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:35.842649 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:35.845614 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:35.846339 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:35.846344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:35.846348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:35.846501 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:35.846941 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:35.846946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:35.846949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:35.847396 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:35.847669 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:35.848923 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0929 02:57:35.849115 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 02:57:35.857023 30527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:35.861672 30527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:35.869149 30527 catalog_manager.cc:1347] Generated new cluster ID: 4584967dfc12481382041c9d3d81e43b
I20260929 02:57:35.869254 30527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:35.894116 30527 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:35.896024 30527 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:35.904080 30527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da9f8e9b46da4445a89bdeda6c565498: Generated new TSK 0
I20260929 02:57:35.904992 30527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:35.909977 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30442
I20260929 02:57:35.911654 30504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:36.000626 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30442
2026-09-29T02:57:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1836 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:36 dist-test-slave-1dkt krb5kdc[30541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:36 dist-test-slave-1dkt krb5kdc[30541](info): set up 2 sockets
Sep 29 02:57:36 dist-test-slave-1dkt krb5kdc[30541](info): commencing operation
krb5kdc: starting...
W20260929 02:57:37.024181 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:37 dist-test-slave-1dkt krb5kdc[30541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:37.259166 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41407
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:41407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:37.572710 30554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:37.572986 30554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:37.573086 30554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:37.580603 30554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:37.580721 30554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:37.580775 30554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:37.580835 30554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:37.580876 30554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:37.590637 30554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:41407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:41407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:37.591794 30554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:37.593040 30554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:37.603788 30559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:37.603840 30562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:37.603820 30560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:37.604446 30554 server_base.cc:1034] running on GCE node
I20260929 02:57:37.605136 30554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:37.606506 30554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:37.607690 30554 hybrid_clock.cc:648] HybridClock initialized: now 1790650657607644 us; error 47 us; skew 500 ppm
I20260929 02:57:37.608032 30554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:37 dist-test-slave-1dkt krb5kdc[30541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650657, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:37.610996 30554 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:37.612382 30554 webserver.cc:460] Webserver started at http://127.8.57.62:36763/ using document root <none> and password file <none>
I20260929 02:57:37.613003 30554 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:37.613098 30554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:37.613336 30554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:37.616266 30554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "b183f424d0624fc6bf7b6d4bdd193d9b"
format_stamp: "Formatted at 2026-09-29 02:57:37 on dist-test-slave-1dkt"
server_key: "464a8eb28f2b4a0462c916ab6fbf922d"
server_key_iv: "ace813c8585e18bffd09bd0b1c241200"
server_key_version: "encryptionkey@0"
I20260929 02:57:37.616967 30554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "b183f424d0624fc6bf7b6d4bdd193d9b"
format_stamp: "Formatted at 2026-09-29 02:57:37 on dist-test-slave-1dkt"
server_key: "464a8eb28f2b4a0462c916ab6fbf922d"
server_key_iv: "ace813c8585e18bffd09bd0b1c241200"
server_key_version: "encryptionkey@0"
I20260929 02:57:37.622012 30554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:57:37.625391 30569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:37.626703 30554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:57:37.626868 30554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "b183f424d0624fc6bf7b6d4bdd193d9b"
format_stamp: "Formatted at 2026-09-29 02:57:37 on dist-test-slave-1dkt"
server_key: "464a8eb28f2b4a0462c916ab6fbf922d"
server_key_iv: "ace813c8585e18bffd09bd0b1c241200"
server_key_version: "encryptionkey@0"
I20260929 02:57:37.627046 30554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:37.640009 30554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:37.643839 30554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:37.644147 30554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:37.661120 30554 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:41407
I20260929 02:57:37.661142 30620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:41407 every 8 connection(s)
I20260929 02:57:37.662449 30554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:37.665994 30621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:37.670620 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30554
I20260929 02:57:37.670755 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:37.671169 8420 external_mini_cluster.cc:1428] Setting key 6c60a498a501602e48e33c814595b807
I20260929 02:57:37.676378 30621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b: Bootstrap starting.
I20260929 02:57:37.679739 30621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:37.680902 30621 log.cc:824] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:37.683736 30621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b: No bootstrap required, opened a new log
I20260929 02:57:37.683995 30624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:37.672854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53873 (local address 127.8.57.62:41407)
0929 02:57:37.673660 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:37.673671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:37.673714 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:57:37.675361 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:37.675381 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:37.675739 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:37.676082 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:37.676666 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:37.677942 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:37.679707 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:37.680443 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:37.680562 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:37.681773 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:37.681783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:37.681800 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:37.682431 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:37.682503 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:37.682530 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:37.682610 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:37.682810 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:57:37.683040 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":348,"threads_started":1}
I20260929 02:57:37.687206 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:37.691229 30621 raft_consensus.cc:348] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41407 } }
I20260929 02:57:37.691679 30621 raft_consensus.cc:374] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:37.691785 30621 raft_consensus.cc:729] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b183f424d0624fc6bf7b6d4bdd193d9b, State: Initialized, Role: FOLLOWER
I20260929 02:57:37.692413 30621 consensus_queue.cc:260] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41407 } }
I20260929 02:57:37.692703 30621 raft_consensus.cc:388] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:37.692840 30621 raft_consensus.cc:482] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:37.693009 30621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:37.695859 30621 raft_consensus.cc:504] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41407 } }
I20260929 02:57:37.696460 30621 leader_election.cc:302] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b183f424d0624fc6bf7b6d4bdd193d9b; no voters:
I20260929 02:57:37.697116 30621 leader_election.cc:288] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:37.697247 30631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:37.697605 30624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:37.691185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35127 (local address 127.8.57.62:41407)
0929 02:57:37.691316 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:37.691321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:37.691749 (+ 428us) server_negotiation.cc:407] Connection header received
0929 02:57:37.691956 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:37.691960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:37.692014 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:37.692098 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:37.692712 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:37.693515 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:37.695212 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:37.695626 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:37.695708 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:37.696764 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:37.696769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:37.696772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:37.696787 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:37.696823 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:37.696837 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:37.696923 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:37.697083 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 02:57:37.697174 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:57:37.698503 30631 raft_consensus.cc:686] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [term 1 LEADER]: Becoming Leader. State: Replica: b183f424d0624fc6bf7b6d4bdd193d9b, State: Running, Role: LEADER
I20260929 02:57:37.699100 30631 consensus_queue.cc:237] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41407 } }
I20260929 02:57:37.699334 30621 sys_catalog.cc:564] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:37.701804 30633 sys_catalog.cc:455] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b183f424d0624fc6bf7b6d4bdd193d9b. Latest consensus state: current_term: 1 leader_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41407 } } }
I20260929 02:57:37.701841 30632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b183f424d0624fc6bf7b6d4bdd193d9b" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 41407 } } }
I20260929 02:57:37.702250 30633 sys_catalog.cc:458] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:37.702250 30632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:37.703130 30641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:37.709030 30641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:37.716990 30641 catalog_manager.cc:1347] Generated new cluster ID: 6d9b57103d3c45b78549060d1c8055f6
I20260929 02:57:37.717096 30641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:37.727977 30641 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:37.729190 30641 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:37.736837 30641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b183f424d0624fc6bf7b6d4bdd193d9b: Generated new TSK 0
I20260929 02:57:37.737584 30641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:37.746584 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30554
W20260929 02:57:37.747666 30616 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53873
W20260929 02:57:37.748382 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30554: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:37.748518 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30554
2026-09-29T02:57:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1747 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:37 dist-test-slave-1dkt krb5kdc[30652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:37 dist-test-slave-1dkt krb5kdc[30652](info): set up 2 sockets
Sep 29 02:57:37 dist-test-slave-1dkt krb5kdc[30652](info): commencing operation
krb5kdc: starting...
W20260929 02:57:38.752218 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:38 dist-test-slave-1dkt krb5kdc[30652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:38.978417 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:39421
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:39421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:39.296006 30665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:39.296286 30665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:39.296386 30665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:39.303458 30665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:39.303568 30665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:39.303622 30665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:39.303680 30665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:39.303723 30665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:39.313292 30665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:39421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:39.314477 30665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:39.315654 30665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:39.326668 30670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:39.326702 30671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:39.326771 30673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:39.327251 30665 server_base.cc:1034] running on GCE node
I20260929 02:57:39.328024 30665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:39.329372 30665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:39.330597 30665 hybrid_clock.cc:648] HybridClock initialized: now 1790650659330553 us; error 35 us; skew 500 ppm
I20260929 02:57:39.330938 30665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:39 dist-test-slave-1dkt krb5kdc[30652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650659, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:39.336553 30665 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:39.337886 30665 webserver.cc:460] Webserver started at http://127.8.57.62:44025/ using document root <none> and password file <none>
I20260929 02:57:39.338521 30665 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:39.338634 30665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:39.338915 30665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:39.341858 30665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "37e1e504875a45a5960398231f914a35"
format_stamp: "Formatted at 2026-09-29 02:57:39 on dist-test-slave-1dkt"
server_key: "9f2080a9ed1f64cef33500b1446baed0"
server_key_iv: "9e2274d44266e316affec3c753ac2e1a"
server_key_version: "encryptionkey@0"
I20260929 02:57:39.342553 30665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "37e1e504875a45a5960398231f914a35"
format_stamp: "Formatted at 2026-09-29 02:57:39 on dist-test-slave-1dkt"
server_key: "9f2080a9ed1f64cef33500b1446baed0"
server_key_iv: "9e2274d44266e316affec3c753ac2e1a"
server_key_version: "encryptionkey@0"
I20260929 02:57:39.347285 30665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:57:39.350555 30680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:39.351750 30665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:57:39.351934 30665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "37e1e504875a45a5960398231f914a35"
format_stamp: "Formatted at 2026-09-29 02:57:39 on dist-test-slave-1dkt"
server_key: "9f2080a9ed1f64cef33500b1446baed0"
server_key_iv: "9e2274d44266e316affec3c753ac2e1a"
server_key_version: "encryptionkey@0"
I20260929 02:57:39.352125 30665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:39.357296 30665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:39.361284 30665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:39.361537 30665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:39.377797 30665 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:39421
I20260929 02:57:39.377835 30731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:39421 every 8 connection(s)
I20260929 02:57:39.379112 30665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:39.380928 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30665
I20260929 02:57:39.381069 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:39.381475 8420 external_mini_cluster.cc:1428] Setting key b50aaa83c7354ee4d91f2a9b6e4184fa
I20260929 02:57:39.383034 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:39.388592 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:39.383114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37445 (local address 127.8.57.62:39421)
0929 02:57:39.383853 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:39.383864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:39.383938 (+ 74us) server_negotiation.cc:407] Connection header received
0929 02:57:39.385440 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:39.385468 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:39.385817 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:39.386144 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:39.387106 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:39.387112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:39.387123 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:39.387140 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:39.387212 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:39.387233 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:39.387380 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:39.387479 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 02:57:39.387637 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":325,"threads_started":1}
I20260929 02:57:39.392133 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:39.395826 30732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35: Bootstrap starting.
I20260929 02:57:39.399187 30732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:39.399227 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:39.396988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38979 (local address 127.8.57.62:39421)
0929 02:57:39.397151 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:39.397157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:39.397598 (+ 441us) server_negotiation.cc:407] Connection header received
0929 02:57:39.397708 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:39.397713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:39.397777 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:39.397874 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:39.398652 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:39.398657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:39.398661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:57:39.398677 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:39.398721 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:57:39.398739 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:39.398851 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:39.398894 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 02:57:39.398944 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:57:39.400417 30732 log.cc:824] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:39.402879 30732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35: No bootstrap required, opened a new log
I20260929 02:57:39.409821 30732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37e1e504875a45a5960398231f914a35" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39421 } }
I20260929 02:57:39.410169 30732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:39.410266 30732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e1e504875a45a5960398231f914a35, State: Initialized, Role: FOLLOWER
I20260929 02:57:39.410813 30732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37e1e504875a45a5960398231f914a35" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39421 } }
I20260929 02:57:39.411041 30732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:39.411193 30732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:39.411377 30732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:39.413517 30732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37e1e504875a45a5960398231f914a35" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39421 } }
I20260929 02:57:39.413981 30732 leader_election.cc:302] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37e1e504875a45a5960398231f914a35; no voters:
I20260929 02:57:39.414520 30732 leader_election.cc:288] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:39.414633 30742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:39.415218 30742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [term 1 LEADER]: Becoming Leader. State: Replica: 37e1e504875a45a5960398231f914a35, State: Running, Role: LEADER
I20260929 02:57:39.415723 30742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37e1e504875a45a5960398231f914a35" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39421 } }
I20260929 02:57:39.416363 30732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:39.418303 30744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37e1e504875a45a5960398231f914a35. Latest consensus state: current_term: 1 leader_uuid: "37e1e504875a45a5960398231f914a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37e1e504875a45a5960398231f914a35" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39421 } } }
I20260929 02:57:39.418684 30744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:39.419236 30743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37e1e504875a45a5960398231f914a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37e1e504875a45a5960398231f914a35" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 39421 } } }
I20260929 02:57:39.419590 30743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:39.421262 30747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:39.426568 30747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:39.434715 30747 catalog_manager.cc:1347] Generated new cluster ID: db5dfdad78be4921b5f3f726b11f9188
I20260929 02:57:39.434828 30747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:39.463346 30747 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:39.463830 30747 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:39.487860 30747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37e1e504875a45a5960398231f914a35: Generated new TSK 0
I20260929 02:57:39.488962 30747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:39.496711 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30665
W20260929 02:57:39.497803 30727 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37445
W20260929 02:57:39.498458 8420 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:57:39.498587 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30665
2026-09-29T02:57:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1750 ms)
[----------] 18 tests from AuthTokenIssuingTest (32158 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:39 dist-test-slave-1dkt krb5kdc[30763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:39 dist-test-slave-1dkt krb5kdc[30763](info): set up 2 sockets
Sep 29 02:57:39 dist-test-slave-1dkt krb5kdc[30763](info): commencing operation
krb5kdc: starting...
W20260929 02:57:40.504261 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:40 dist-test-slave-1dkt krb5kdc[30763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:40.740018 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40829
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40829,127.8.57.61:40067,127.8.57.60:34325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:41.052047 30776 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:57:41.052366 30776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:41.052436 30776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:41.052517 30776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:41.059803 30776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:41.059914 30776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:41.059980 30776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:41.060030 30776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:41.060089 30776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:41.069748 30776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40829,127.8.57.61:40067,127.8.57.60:34325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:40829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:41.070979 30776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:41.072413 30776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:41.083456 30781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:41.083391 30784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:41.083424 30782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:41.084323 30776 server_base.cc:1034] running on GCE node
I20260929 02:57:41.084997 30776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:41.086236 30776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:41.087448 30776 hybrid_clock.cc:648] HybridClock initialized: now 1790650661087386 us; error 39 us; skew 500 ppm
I20260929 02:57:41.087759 30776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:41 dist-test-slave-1dkt krb5kdc[30763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:41.090831 30776 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:41.092224 30776 webserver.cc:460] Webserver started at http://127.8.57.62:36589/ using document root <none> and password file <none>
I20260929 02:57:41.092860 30776 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:41.092944 30776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:41.093189 30776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:41.096047 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "5e8973e8f9b9413e9f15490dba4ccecc"
format_stamp: "Formatted at 2026-09-29 02:57:41 on dist-test-slave-1dkt"
server_key: "969d637c4251ecea9e56c858bc7e1ad6"
server_key_iv: "71cd00cdbd447357c7414a9c48ce9185"
server_key_version: "encryptionkey@0"
I20260929 02:57:41.096732 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "5e8973e8f9b9413e9f15490dba4ccecc"
format_stamp: "Formatted at 2026-09-29 02:57:41 on dist-test-slave-1dkt"
server_key: "969d637c4251ecea9e56c858bc7e1ad6"
server_key_iv: "71cd00cdbd447357c7414a9c48ce9185"
server_key_version: "encryptionkey@0"
I20260929 02:57:41.101418 30776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:57:41.104519 30791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:41.105770 30776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:41.105922 30776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "5e8973e8f9b9413e9f15490dba4ccecc"
format_stamp: "Formatted at 2026-09-29 02:57:41 on dist-test-slave-1dkt"
server_key: "969d637c4251ecea9e56c858bc7e1ad6"
server_key_iv: "71cd00cdbd447357c7414a9c48ce9185"
server_key_version: "encryptionkey@0"
I20260929 02:57:41.106114 30776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:41.123804 30776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:41.127710 30776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:41.127988 30776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:41.149637 30776 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:40829
I20260929 02:57:41.149657 30842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:40829 every 8 connection(s)
I20260929 02:57:41.150861 30776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:41.151211 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30776
I20260929 02:57:41.151418 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:41.152187 8420 external_mini_cluster.cc:1428] Setting key bcb74956687bc6c0b47ce272965430fc
I20260929 02:57:41.155452 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:41.164678 30843 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.57.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.61@KRBTEST.COM" created.
Sep 29 02:57:41 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:41.186820 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.168799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.8.57.62:40829)
0929 02:57:41.169440 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:41.169446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:41.169484 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:57:41.171167 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:41.171172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:41.171429 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:41.171576 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:41.173039 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.174177 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.176198 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.176839 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.176981 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.180239 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:41.180247 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:41.180259 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:41.180288 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:41.182781 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.183401 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.183407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.183409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.183553 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.183936 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.183939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.183942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.184404 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:41.184592 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:41.185079 (+ 487us) server_negotiation.cc:299] Negotiation successful
0929 02:57:41.185425 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":462,"thread_start_us":305,"threads_started":1}
I20260929 02:57:41.186838 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.167583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40829 (local address 127.0.0.1:55758)
0929 02:57:41.168450 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.168498 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:41.170798 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:41.171649 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:41.171655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:41.172383 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:41.172838 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.172864 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.174334 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.174340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.176047 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.176061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.177077 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.177084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.177221 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.177842 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:41.177865 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:41.180030 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:41.182897 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.182908 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.182921 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.183281 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.183655 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.183659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.183663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.183816 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.184531 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:41.184539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:41.184729 (+ 190us) client_negotiation.cc:769] Sending connection context
0929 02:57:41.184937 (+ 208us) client_negotiation.cc:240] Negotiation successful
0929 02:57:41.185541 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":136,"thread_start_us":335,"threads_started":1}
I20260929 02:57:41.190399 30843 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:41.191431 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.190993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57548)
0929 02:57:41.191155 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.191270 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 02:57:41.191666 30792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
I20260929 02:57:41.193194 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.192807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57550)
0929 02:57:41.192960 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.193051 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 02:57:41.193369 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:40067: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab.
Entry for principal kudu/127.8.57.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab.
I20260929 02:57:41.206905 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:40067
--webserver_interface=127.8.57.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab
--principal=kudu/127.8.57.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40829,127.8.57.61:40067,127.8.57.60:34325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:57:41.242669 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } attempt: 1
I20260929 02:57:41.243768 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.243338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57554)
0929 02:57:41.243534 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.243629 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 02:57:41.244740 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.244422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57560)
0929 02:57:41.244550 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.244617 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:57:41.244920 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:40067: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
I20260929 02:57:41.313128 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } attempt: 2
I20260929 02:57:41.314303 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.313835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57574)
0929 02:57:41.314079 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.314168 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 02:57:41.315275 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.314967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57582)
0929 02:57:41.315075 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.315151 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 02:57:41.315459 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:40067: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
I20260929 02:57:41.406688 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } attempt: 3
I20260929 02:57:41.407845 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.407429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57592)
0929 02:57:41.407619 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.407711 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 02:57:41.408901 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.408596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57594)
0929 02:57:41.408707 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.408773 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:57:41.409106 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:40067: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
W20260929 02:57:41.524869 30850 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:57:41.525177 30850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:41.525267 30850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:41.525372 30850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:41.532546 30850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:41.532637 30850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:41.532702 30850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:41.532765 30850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:41.532845 30850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:41.542440 30850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40829,127.8.57.61:40067,127.8.57.60:34325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab
--principal=kudu/127.8.57.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:40067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.57.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:41.543670 30850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:41.545020 30850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:57:41.552295 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } attempt: 4
I20260929 02:57:41.553372 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.552895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57608)
0929 02:57:41.553171 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.553252 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260929 02:57:41.554260 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.553956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57610)
0929 02:57:41.554075 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.554150 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:57:41.554503 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:40067: Network error: Client connection negotiation failed: client connection to 127.8.57.61:40067: connect: Connection refused (error 111)
W20260929 02:57:41.556013 30855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:41.556064 30856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:41.556612 30858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:41.557278 30850 server_base.cc:1034] running on GCE node
I20260929 02:57:41.557999 30850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:41.559295 30850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:41.560523 30850 hybrid_clock.cc:648] HybridClock initialized: now 1790650661560451 us; error 41 us; skew 500 ppm
I20260929 02:57:41.560813 30850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:41 dist-test-slave-1dkt krb5kdc[30763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:41.563684 30850 init.cc:377] Logged in from keytab as kudu/127.8.57.61@KRBTEST.COM (short username kudu)
I20260929 02:57:41.565033 30850 webserver.cc:460] Webserver started at http://127.8.57.61:33833/ using document root <none> and password file <none>
I20260929 02:57:41.565685 30850 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:41.565784 30850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:41.566082 30850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:41.569074 30850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data/instance:
uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7"
format_stamp: "Formatted at 2026-09-29 02:57:41 on dist-test-slave-1dkt"
server_key: "06a1b3c2ef18ae6d9d01add9145109d4"
server_key_iv: "527755e1b889b7032cafb2bc4b0f2ba1"
server_key_version: "encryptionkey@0"
I20260929 02:57:41.569830 30850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal/instance:
uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7"
format_stamp: "Formatted at 2026-09-29 02:57:41 on dist-test-slave-1dkt"
server_key: "06a1b3c2ef18ae6d9d01add9145109d4"
server_key_iv: "527755e1b889b7032cafb2bc4b0f2ba1"
server_key_version: "encryptionkey@0"
I20260929 02:57:41.574700 30850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260929 02:57:41.577925 30865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:41.579212 30850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:57:41.579357 30850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal
uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7"
format_stamp: "Formatted at 2026-09-29 02:57:41 on dist-test-slave-1dkt"
server_key: "06a1b3c2ef18ae6d9d01add9145109d4"
server_key_iv: "527755e1b889b7032cafb2bc4b0f2ba1"
server_key_version: "encryptionkey@0"
I20260929 02:57:41.579562 30850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:41.607499 30850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:41.611289 30850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:41.611548 30850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:41.628129 30850 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.61:40067
I20260929 02:57:41.628167 30916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.61:40067 every 8 connection(s)
I20260929 02:57:41.629454 30850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
I20260929 02:57:41.633013 30917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:41.639286 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30850
I20260929 02:57:41.639451 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-1/wal/instance
I20260929 02:57:41.639842 8420 external_mini_cluster.cc:1428] Setting key 2c8b99e8c5328447b72b87f33e7b23fe
I20260929 02:57:41.639223 30917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:57:41 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:41.658656 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.641742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.8.57.62:40829)
0929 02:57:41.641962 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:41.641969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:41.642934 (+ 965us) server_negotiation.cc:407] Connection header received
0929 02:57:41.645162 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:41.645167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:41.645233 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:41.645310 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:41.647073 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.647732 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.649493 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.649955 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.650055 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.653416 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:41.653420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:41.653424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:41.653458 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:41.655898 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.656643 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.656647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.656650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.656759 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.657147 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.657149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.657151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.657411 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:41.657522 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:41.658262 (+ 740us) server_negotiation.cc:299] Negotiation successful
0929 02:57:41.658419 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:57:41.660012 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.642058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40829 (local address 127.0.0.1:55772)
0929 02:57:41.642775 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.642817 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:41.644877 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:41.645545 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:41.645555 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:41.646205 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:41.646914 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.646936 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.647900 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.647906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.649330 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.649341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.650127 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.650131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.650324 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.650934 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:41.650955 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:41.653188 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:41.656019 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.656027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.656043 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.656548 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.656842 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.656847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.656850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.657061 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.657513 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:41.657520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:41.657903 (+ 383us) client_negotiation.cc:769] Sending connection context
0929 02:57:41.658192 (+ 289us) client_negotiation.cc:240] Negotiation successful
0929 02:57:41.658670 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":267,"threads_started":1}
I20260929 02:57:41.661693 30917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.57.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.60@KRBTEST.COM" created.
Sep 29 02:57:41 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:41.674482 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.662375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57624)
0929 02:57:41.662543 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.662560 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:41.662634 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:41.663658 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:41.663663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:41.664129 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:41.664294 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.664305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.665580 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.665584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.666435 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.666447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.667211 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.667217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.667385 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.667855 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:41.667894 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:41.669703 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:41.672442 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.672447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.672451 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.672779 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.673109 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.673113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.673115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.673237 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.673827 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:41.673830 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:41.673954 (+ 124us) client_negotiation.cc:769] Sending connection context
0929 02:57:41.674054 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:57:41.674229 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 02:57:41.674738 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.662692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57624 (local address 127.8.57.61:40067)
0929 02:57:41.663276 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:41.663281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:41.663304 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:57:41.663343 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:41.663348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:41.663501 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:41.663606 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:41.664451 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.665448 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.666551 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.667088 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.667192 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.669874 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:41.669884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:41.669894 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:41.669920 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:41.672329 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.672885 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.672891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.672893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.672994 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.673325 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.673329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.673332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.673715 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:41.673911 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:41.674196 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 02:57:41.674369 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":341,"threads_started":1}
I20260929 02:57:41.676826 30917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:41.677781 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.677394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43096)
0929 02:57:41.677539 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.677639 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 02:57:41.677995 30869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:41.679302 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.679016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43098)
0929 02:57:41.679142 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.679212 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:57:41.679484 30917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab.
Entry for principal kudu/127.8.57.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab.
I20260929 02:57:41.695173 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:34325
--webserver_interface=127.8.57.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab
--principal=kudu/127.8.57.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:44291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:40829,127.8.57.61:40067,127.8.57.60:34325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:57:41.728754 30917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 1
I20260929 02:57:41.729806 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.729368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43100)
0929 02:57:41.729558 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.729640 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 02:57:41.730731 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.730403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43112)
0929 02:57:41.730547 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.730629 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:57:41.730912 30917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:41.799106 30917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 2
I20260929 02:57:41.800195 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.799757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43128)
0929 02:57:41.799958 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.800046 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 02:57:41.801100 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.800792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43140)
0929 02:57:41.800916 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.800985 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 02:57:41.801281 30917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:41.853744 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } attempt: 5
Sep 29 02:57:41 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:41.866930 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.854494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57628)
0929 02:57:41.854706 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.854728 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:41.854870 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:41.855217 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:41.855222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:41.855640 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:41.855821 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.855830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.856852 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.856856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.857689 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:41.857701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.858381 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.858387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:41.858570 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.859166 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:41.859182 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:41.861261 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:41.864819 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.864822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.864825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.865173 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.865539 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:41.865542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:41.865545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:41.865659 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:41.866253 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:41.866256 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:41.866378 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 02:57:41.866494 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 02:57:41.866687 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 02:57:41.867146 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.854547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.8.57.61:40067)
0929 02:57:41.854737 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:41.854742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:41.854812 (+ 70us) server_negotiation.cc:407] Connection header received
0929 02:57:41.854996 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:41.855001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:41.855079 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:41.855163 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:41.855991 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.856701 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.857838 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:41.858239 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:41.858346 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:41.861457 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:41.861464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:41.861469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:41.861511 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:41.864691 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.865286 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.865292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.865295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.865444 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:41.865752 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:41.865757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:41.865760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:41.866147 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:41.866297 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:41.866614 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 02:57:41.866831 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 02:57:41.868139 30843 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:41.869212 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.868860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43146)
0929 02:57:41.869011 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.869083 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:57:41.870066 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.869773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43152)
0929 02:57:41.869898 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.869967 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:57:41.870236 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:41.892479 30917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 3
I20260929 02:57:41.893520 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.893061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43156)
0929 02:57:41.893274 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.893359 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 02:57:41.894479 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.894081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43166)
0929 02:57:41.894292 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.894365 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260929 02:57:41.894672 30917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:41.921434 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 1
I20260929 02:57:41.922452 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.921994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43172)
0929 02:57:41.922175 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.922287 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 02:57:41.923614 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.923230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43176)
0929 02:57:41.923387 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.923481 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 02:57:41.923828 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:41.992051 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 2
I20260929 02:57:41.993168 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.992617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43182)
0929 02:57:41.992854 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.992974 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 02:57:41.994144 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:41.993787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43184)
0929 02:57:41.993926 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:41.994011 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:57:41.994349 30843 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
W20260929 02:57:42.006579 30924 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:57:42.006861 30924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:42.006933 30924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:42.007012 30924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:42.014184 30924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:42.014276 30924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:42.014333 30924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:42.014380 30924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:42.014443 30924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:42.023952 30924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:44291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:40829,127.8.57.61:40067,127.8.57.60:34325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab
--principal=kudu/127.8.57.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:34325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.57.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:42.025117 30924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:42.026305 30924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:42.037128 30929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:42.037156 30932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:42.037091 30930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:42.037927 30924 server_base.cc:1034] running on GCE node
I20260929 02:57:42.037895 30917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 4
I20260929 02:57:42.038645 30924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:42.038897 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.038443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43200)
0929 02:57:42.038649 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.038728 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 02:57:42.039832 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.039505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43202)
0929 02:57:42.039660 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.039730 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:57:42.040031 30924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260929 02:57:42.040074 30917 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:34325: Network error: Client connection negotiation failed: client connection to 127.8.57.60:34325: connect: Connection refused (error 111)
I20260929 02:57:42.041306 30924 hybrid_clock.cc:648] HybridClock initialized: now 1790650662041204 us; error 45 us; skew 500 ppm
I20260929 02:57:42.041620 30924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:42.044593 30924 init.cc:377] Logged in from keytab as kudu/127.8.57.60@KRBTEST.COM (short username kudu)
I20260929 02:57:42.045964 30924 webserver.cc:460] Webserver started at http://127.8.57.60:45451/ using document root <none> and password file <none>
I20260929 02:57:42.046591 30924 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:42.046685 30924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:42.046967 30924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:42.049773 30924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data/instance:
uuid: "b214d8654c6248dd96a34da3d38e224b"
format_stamp: "Formatted at 2026-09-29 02:57:42 on dist-test-slave-1dkt"
server_key: "8cbf086dd767d1b4bbb68a2233ab91c6"
server_key_iv: "889dd954fa7f177ab20028459dc47b0b"
server_key_version: "encryptionkey@0"
I20260929 02:57:42.050491 30924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal/instance:
uuid: "b214d8654c6248dd96a34da3d38e224b"
format_stamp: "Formatted at 2026-09-29 02:57:42 on dist-test-slave-1dkt"
server_key: "8cbf086dd767d1b4bbb68a2233ab91c6"
server_key_iv: "889dd954fa7f177ab20028459dc47b0b"
server_key_version: "encryptionkey@0"
I20260929 02:57:42.055205 30924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:57:42.058290 30939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:42.059484 30924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:57:42.059626 30924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal
uuid: "b214d8654c6248dd96a34da3d38e224b"
format_stamp: "Formatted at 2026-09-29 02:57:42 on dist-test-slave-1dkt"
server_key: "8cbf086dd767d1b4bbb68a2233ab91c6"
server_key_iv: "889dd954fa7f177ab20028459dc47b0b"
server_key_version: "encryptionkey@0"
I20260929 02:57:42.059806 30924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:42.072808 30924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:42.076627 30924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:42.076849 30924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:42.092363 30924 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.60:34325
I20260929 02:57:42.092381 30990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.60:34325 every 8 connection(s)
I20260929 02:57:42.093575 30924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
I20260929 02:57:42.094758 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 30924
I20260929 02:57:42.094931 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650321430198-8420-0/minicluster-data/master-2/wal/instance
I20260929 02:57:42.095453 8420 external_mini_cluster.cc:1428] Setting key a6952247fd4dfb9e919ca0081981bbec
I20260929 02:57:42.097757 30991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:42.100556 30843 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 3
I20260929 02:57:42.106693 30991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
I20260929 02:57:42.117523 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.101166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43222)
0929 02:57:42.101367 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.101383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:42.101473 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:42.102385 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:42.102390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:42.102877 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:42.103053 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.103065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.104529 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.104533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.105402 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.105426 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.106387 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.106393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.106587 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.107167 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:42.107189 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:42.109827 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:42.115262 (+ 5435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.115267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.115270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.115672 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.116038 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.116043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.116047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.116171 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.116761 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:42.116766 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:42.116948 (+ 182us) client_negotiation.cc:769] Sending connection context
0929 02:57:42.117068 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 02:57:42.117251 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 02:57:42.118923 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.101198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.8.57.60:34325)
0929 02:57:42.102056 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.102061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.102085 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:57:42.102128 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.102140 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.102206 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.102320 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.103240 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.104394 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.105552 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.106216 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.106876 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.110151 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.110161 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.110182 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.110229 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.115142 (+ 4913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.115770 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.115775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.115778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.115940 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.116266 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.116271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.116275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.116614 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.117855 (+ 1241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.117974 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.118112 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":569,"threads_started":1}
I20260929 02:57:42.118983 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.097209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.8.57.60:34325)
0929 02:57:42.097894 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.097910 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.097969 (+ 59us) server_negotiation.cc:407] Connection header received
0929 02:57:42.099566 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.099586 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.099987 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.100348 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.101919 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.103625 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.105418 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.106136 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.106291 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.110790 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.110798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.110802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.110836 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.114684 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.115384 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.115393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.115400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.115583 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.115962 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.115967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.115970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.116654 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.116918 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.117332 (+ 414us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.117720 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":494,"spinlock_wait_cycles":15616,"thread_start_us":278,"threads_started":1}
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:42.126360 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.109393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:40829 (local address 127.0.0.1:55778)
0929 02:57:42.110453 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.110490 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:42.110687 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:42.111005 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:42.111012 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:42.111908 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:42.112224 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.112234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.113620 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.113624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.114718 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.114730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.116057 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.116063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.116314 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.117737 (+ 1423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:42.117757 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:42.120434 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:42.123817 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.123828 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.123834 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.124331 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.124707 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.124713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.124717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.124862 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.125381 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:42.125389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:42.125543 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 02:57:42.125680 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 02:57:42.125967 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":61,"thread_start_us":714,"threads_started":1}
I20260929 02:57:42.126394 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.109573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.8.57.62:40829)
0929 02:57:42.109755 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.109760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.110571 (+ 811us) server_negotiation.cc:407] Connection header received
0929 02:57:42.110798 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.110804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.110861 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.110934 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.112436 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.113418 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.114932 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.115584 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.115699 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.120631 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.120637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.120642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.120680 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.123684 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.124437 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.124442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.124446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.124604 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.124967 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.124972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.124976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.125251 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.125387 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.125842 (+ 455us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.126041 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:57:42.126564 30843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc: Bootstrap starting.
I20260929 02:57:42.127692 30991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:42.131601 30843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc: Neither blocks nor log segments found. Creating new log.
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:42.132864 30843 log.cc:824] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:42.136303 30843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc: No bootstrap required, opened a new log
I20260929 02:57:42.138923 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.122054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55794 (local address 127.8.57.62:40829)
0929 02:57:42.122788 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.122794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.122816 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:57:42.122871 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.122875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.122943 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.123957 (+ 1014us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.124576 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.125735 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.127621 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.128349 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.129011 (+ 662us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.133307 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.133314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.133319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.133359 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.136440 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.136999 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.137004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.137008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.137119 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.137511 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.137514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.137516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.137788 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.138052 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.138318 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.138528 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":480,"threads_started":1}
I20260929 02:57:42.141256 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.128233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:40067 (local address 127.0.0.1:57640)
0929 02:57:42.128466 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.128483 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:42.128569 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:42.129209 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:42.129214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:42.129576 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:42.129704 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.129712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.130696 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.130702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.131650 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.131662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.132457 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.132463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.132632 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.133108 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:42.133123 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:42.135301 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:42.138962 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.138967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.138971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.139355 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.139700 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.139705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.139709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.139833 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.140390 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:42.140395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:42.140542 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 02:57:42.140656 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 02:57:42.140914 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260929 02:57:42.141345 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.128543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.8.57.61:40067)
0929 02:57:42.128759 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.128765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.128781 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:57:42.128841 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.128846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.128914 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.129013 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.129846 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.130573 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.131785 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.132309 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.132412 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.135488 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.135494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.135497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.135536 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.138832 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.139466 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.139471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.139475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.139601 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.139968 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.139973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.139977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.140277 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.140410 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.140768 (+ 358us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.140974 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 02:57:42.142561 30991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:42.146592 30843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.147001 30843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e8973e8f9b9413e9f15490dba4ccecc, State: Initialized, Role: FOLLOWER
I20260929 02:57:42.147526 30843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:42.154824 31000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.155210 31000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:42.156992 30843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:42.157476 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.143479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.8.57.60:34325)
0929 02:57:42.143611 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.143616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.143632 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:57:42.143673 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.143678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.143736 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.143846 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.144673 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.145716 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.147309 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.147946 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.148054 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.151453 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.151459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.151464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.151496 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.154598 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.155424 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.155429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.155433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.155575 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.155971 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.155975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.155978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.156336 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.156452 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.156933 (+ 481us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.157147 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:57:42.158514 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.143292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43238)
0929 02:57:42.143415 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.143428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:42.143513 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:42.143852 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:42.143855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:42.144312 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:42.144529 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.144541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.145861 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.145865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.147171 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.147185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.148171 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.148175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.148307 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.148844 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:42.148867 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:42.151267 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:42.154726 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.154730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.154734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.155316 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.155696 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.155699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.155703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.155846 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.156535 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:42.156539 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:42.156664 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 02:57:42.157896 (+ 1232us) client_negotiation.cc:240] Negotiation successful
0929 02:57:42.158170 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 02:57:42.159528 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.140970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.8.57.61:40067)
0929 02:57:42.141508 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.141512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.142303 (+ 791us) server_negotiation.cc:407] Connection header received
0929 02:57:42.142446 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.142451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.142521 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.142613 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.145096 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.145802 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.147032 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.147660 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.147777 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.153228 (+ 5451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.153234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.153238 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.153275 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.156397 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.157127 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.157132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.157136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.157302 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.157739 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.157744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.157747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.158045 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.158277 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.158793 (+ 516us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.158987 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":823,"threads_started":1}
I20260929 02:57:42.164798 30991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: Bootstrap starting.
I20260929 02:57:42.167951 30991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:42.168710 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:42.169106 30991 log.cc:824] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:42.171747 30991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: No bootstrap required, opened a new log
I20260929 02:57:42.172200 30818 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5e8973e8f9b9413e9f15490dba4ccecc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55794
I20260929 02:57:42.172508 30818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 02:57:42.172668 30818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:42.174717 30818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.175830 30818 leader_election.cc:288] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [CANDIDATE]: Term 1 election: Requested vote from peers 9c7f0e2a52f4482d9383ac26ebb6aea7 (127.8.57.61:40067), b214d8654c6248dd96a34da3d38e224b (127.8.57.60:34325)
I20260929 02:57:42.180195 30991 raft_consensus.cc:348] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.180595 30991 raft_consensus.cc:729] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b214d8654c6248dd96a34da3d38e224b, State: Initialized, Role: FOLLOWER
I20260929 02:57:42.181087 30991 consensus_queue.cc:260] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.182264 31015 sys_catalog.cc:455] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.183131 31015 sys_catalog.cc:458] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:42.183362 30991 sys_catalog.cc:564] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:42.183454 30966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b214d8654c6248dd96a34da3d38e224b"
I20260929 02:57:42.183921 30966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:42.184787 31012 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 02:57:42.187414 30966 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e8973e8f9b9413e9f15490dba4ccecc in term 1.
I20260929 02:57:42.188153 30795 leader_election.cc:302] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e8973e8f9b9413e9f15490dba4ccecc, b214d8654c6248dd96a34da3d38e224b; no voters:
I20260929 02:57:42.189021 31000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:42.194141 31000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [term 1 LEADER]: Becoming Leader. State: Replica: 5e8973e8f9b9413e9f15490dba4ccecc, State: Running, Role: LEADER
I20260929 02:57:42.194741 31000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.199165 31018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e8973e8f9b9413e9f15490dba4ccecc. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.199558 31018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:42.189725 30776 master_runner.cc:186] Error getting master registration for 127.8.57.61:40067: OK, instance_id { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" instance_seqno: 1790650661623352 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:57:42.210726 31024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:42.210981 30776 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:57:42.230167 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.208619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.8.57.60:34325)
0929 02:57:42.208789 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.208795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.211505 (+ 2710us) server_negotiation.cc:407] Connection header received
0929 02:57:42.211946 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.211952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.212038 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.212135 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.214962 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.216087 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.218899 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.219512 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.219598 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.223506 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.223512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.223517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.223553 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.227013 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.227825 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.227835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.227839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.228023 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.228457 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.228461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.228464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.228833 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.229071 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.229607 (+ 536us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.229817 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:57:42.231040 31024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:42.233448 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.210856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57670 (local address 127.8.57.61:40067)
0929 02:57:42.211058 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.211063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.212293 (+ 1230us) server_negotiation.cc:407] Connection header received
0929 02:57:42.212682 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.212689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.212771 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.212842 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.216628 (+ 3786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.217778 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.219822 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.220520 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.220614 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.225307 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.225315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.225320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.225366 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.229192 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.230537 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.230543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.230545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.230726 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.231338 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.231343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.231346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.231697 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.231969 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.232775 (+ 806us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.233110 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 02:57:42.239804 30966 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [term 1 FOLLOWER]: Refusing update from remote peer 5e8973e8f9b9413e9f15490dba4ccecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:42.240808 31000 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:42.245832 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.200052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.8.57.62:40829)
0929 02:57:42.200280 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.200287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.212630 (+ 12343us) server_negotiation.cc:407] Connection header received
0929 02:57:42.212879 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.212884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.212954 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.213019 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.215141 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.216131 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.218021 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.218650 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.218744 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.234755 (+ 16011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.234763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.234769 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.234815 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.239118 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.240283 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.240290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.240294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.240646 (+ 352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.241729 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.241763 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.241766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.242500 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.244517 (+ 2017us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.244763 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.245165 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 02:57:42.259271 31015 sys_catalog.cc:455] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e8973e8f9b9413e9f15490dba4ccecc. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.259615 31015 sys_catalog.cc:458] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:42.260880 31018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.261140 31018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:42.261390 31033 mvcc.cc:205] Tried to move back new op lower bound from 7334505112524673024 to 7334505112354869248. Current Snapshot: MvccSnapshot[applied={T|T < 7334505112524673024}]
I20260929 02:57:42.262414 31024 catalog_manager.cc:1347] Generated new cluster ID: 011f4ca656194ae794973c32d4c562a9
I20260929 02:57:42.262523 31024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:42.268188 30924 master_runner.cc:186] Error getting master registration for 127.8.57.61:40067: OK, instance_id { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" instance_seqno: 1790650661623352 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:57:42.278314 30924 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:57:42.278818 31015 sys_catalog.cc:455] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.279074 31015 sys_catalog.cc:458] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:42.286055 31024 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:42.287719 31039 catalog_manager.cc:1259] Loaded cluster ID: 011f4ca656194ae794973c32d4c562a9
I20260929 02:57:42.287806 31039 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: loading cluster ID for follower catalog manager: success
I20260929 02:57:42.287904 31024 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:42.291303 31039 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: acquiring CA information for follower catalog manager: success
W20260929 02:57:42.292598 31039 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260929 02:57:42.309444 31024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc: Generated new TSK 0
I20260929 02:57:42.310704 31024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:42.339222 30917 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } attempt: 5
Sep 29 02:57:42 dist-test-slave-1dkt krb5kdc[30763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
I20260929 02:57:42.357822 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.339817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:34325 (local address 127.0.0.1:43266)
0929 02:57:42.340035 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:42.340057 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:42.340172 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:42.340610 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:42.340615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:42.341121 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:42.341328 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.341342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.342858 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.342864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.344443 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:42.344459 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.345621 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.345627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:42.345834 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.346455 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:42.346478 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:42.349207 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:42.354424 (+ 5217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.354430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.354435 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.354947 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.355558 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:42.355563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:42.355578 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:42.355740 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:42.356735 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:42.356741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:42.356930 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 02:57:42.357121 (+ 191us) client_negotiation.cc:240] Negotiation successful
0929 02:57:42.357446 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 02:57:42.359547 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:42.340161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.8.57.60:34325)
0929 02:57:42.340313 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.340319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.340336 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:57:42.340391 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.340396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.340477 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.340577 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.341459 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.342675 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.344615 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:42.345315 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:42.345436 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:42.349412 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:42.349419 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:42.349424 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:42.349463 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:42.354240 (+ 4777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.355130 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.355136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.355140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.355399 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:42.355951 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:42.355957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:42.355987 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:42.356538 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:42.358316 (+ 1778us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:42.358554 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 02:57:42.358940 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:57:42.365631 30917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: Bootstrap starting.
I20260929 02:57:42.370888 30917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:42.372864 30917 log.cc:824] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:42.381671 30917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: No bootstrap required, opened a new log
I20260929 02:57:42.398223 30917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.398838 30917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7f0e2a52f4482d9383ac26ebb6aea7, State: Initialized, Role: FOLLOWER
I20260929 02:57:42.399586 30917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } }
I20260929 02:57:42.412057 31045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.412513 31045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:42.412698 30892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7"
I20260929 02:57:42.413086 30891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:42.413319 30917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:42.416993 30891 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [term 1 FOLLOWER]: Refusing update from remote peer 5e8973e8f9b9413e9f15490dba4ccecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:42.418128 31000 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:42.420411 30892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e8973e8f9b9413e9f15490dba4ccecc in term 1.
W20260929 02:57:42.444501 30997 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.435705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.8.57.62:40829)
0929 02:57:42.435920 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.435926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.442435 (+ 6509us) server_negotiation.cc:407] Connection header received
0929 02:57:42.442904 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.442911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.442996 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.443066 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.444228 (+ 1162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55818: BlockingRecv error: recv got EOF from 127.0.0.1:55818 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 02:57:42.446620 30994 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.435348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.8.57.60:34325)
0929 02:57:42.435545 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.435552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.445011 (+ 9459us) server_negotiation.cc:407] Connection header received
0929 02:57:42.445235 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.445241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.445331 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.445410 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.446338 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43282: BlockingRecv error: recv got EOF from 127.0.0.1:43282 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 02:57:42.456543 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.454171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.8.57.61:40067)
0929 02:57:42.454343 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.454358 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.454930 (+ 572us) server_negotiation.cc:407] Connection header received
0929 02:57:42.455144 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.455149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.455241 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.455306 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.456305 (+ 999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57672: BlockingRecv error: recv got EOF from 127.0.0.1:57672 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:57:42.460546 31045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e8973e8f9b9413e9f15490dba4ccecc. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.460876 31045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:42.460527 31058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:42.460994 31058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 02:57:42.498692 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.485327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.8.57.61:40067)
0929 02:57:42.485530 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.485537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.496503 (+ 10966us) server_negotiation.cc:407] Connection header received
0929 02:57:42.497047 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.497053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.497209 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.497289 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.498386 (+ 1097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57682: BlockingRecv error: recv got EOF from 127.0.0.1:57682 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 02:57:42.501714 31072 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260929 02:57:42.519699 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.517839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57694 (local address 127.8.57.61:40067)
0929 02:57:42.518013 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.518019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.518479 (+ 460us) server_negotiation.cc:407] Connection header received
0929 02:57:42.518649 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.518654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.518724 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.518796 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.519506 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57694: BlockingRecv error: recv got EOF from 127.0.0.1:57694 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260929 02:57:42.529665 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.524118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.8.57.61:40067)
0929 02:57:42.524270 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.524276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.528421 (+ 4145us) server_negotiation.cc:407] Connection header received
0929 02:57:42.528633 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.528639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.528716 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.528795 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.529437 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57700: BlockingRecv error: recv got EOF from 127.0.0.1:57700 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260929 02:57:42.538028 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.536187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.8.57.61:40067)
0929 02:57:42.536410 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.536416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.536836 (+ 420us) server_negotiation.cc:407] Connection header received
0929 02:57:42.537000 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.537006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.537085 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.537161 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.537815 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57706: BlockingRecv error: recv got EOF from 127.0.0.1:57706 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260929 02:57:42.551023 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.549161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57716 (local address 127.8.57.61:40067)
0929 02:57:42.549353 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.549359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.549813 (+ 454us) server_negotiation.cc:407] Connection header received
0929 02:57:42.549979 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.549983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.550047 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.550126 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.550794 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57716: BlockingRecv error: recv got EOF from 127.0.0.1:57716 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 02:57:42.572306 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.570089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.8.57.61:40067)
0929 02:57:42.570288 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.570294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.570787 (+ 493us) server_negotiation.cc:407] Connection header received
0929 02:57:42.571067 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.571071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.571145 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.571263 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.572084 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57728: BlockingRecv error: recv got EOF from 127.0.0.1:57728 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:57:42.609705 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.607583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.8.57.61:40067)
0929 02:57:42.607777 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.607784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.608318 (+ 534us) server_negotiation.cc:407] Connection header received
0929 02:57:42.608513 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.608517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.608589 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.608671 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.609453 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57732: BlockingRecv error: recv got EOF from 127.0.0.1:57732 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260929 02:57:42.678912 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.676895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.8.57.61:40067)
0929 02:57:42.677098 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.677104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.677573 (+ 469us) server_negotiation.cc:407] Connection header received
0929 02:57:42.677760 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.677764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.677836 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.677920 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.678692 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57736: BlockingRecv error: recv got EOF from 127.0.0.1:57736 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 02:57:42.811858 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:42.810088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.8.57.61:40067)
0929 02:57:42.810264 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:42.810271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:42.810647 (+ 376us) server_negotiation.cc:407] Connection header received
0929 02:57:42.810829 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:42.810833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:42.810904 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:42.810995 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:42.811674 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57750: BlockingRecv error: recv got EOF from 127.0.0.1:57750 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:57:43.035324 31000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8973e8f9b9413e9f15490dba4ccecc" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 40829 } } peers { permanent_uuid: "9c7f0e2a52f4482d9383ac26ebb6aea7" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 40067 } } peers { permanent_uuid: "b214d8654c6248dd96a34da3d38e224b" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 34325 } } }
I20260929 02:57:43.035640 31000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8973e8f9b9413e9f15490dba4ccecc [sys.catalog]: This master's current role is: LEADER
W20260929 02:57:43.073521 30921 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:43.071958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.8.57.61:40067)
0929 02:57:43.072137 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:43.072142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:43.072418 (+ 276us) server_negotiation.cc:407] Connection header received
0929 02:57:43.072582 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:43.072585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:43.072658 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:43.072748 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:43.073346 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57762: BlockingRecv error: recv got EOF from 127.0.0.1:57762 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:57:43.295256 31039 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b214d8654c6248dd96a34da3d38e224b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:57:43.506071 31072 catalog_manager.cc:1259] Loaded cluster ID: 011f4ca656194ae794973c32d4c562a9
I20260929 02:57:43.506184 31072 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: loading cluster ID for follower catalog manager: success
I20260929 02:57:43.508883 31072 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: acquiring CA information for follower catalog manager: success
I20260929 02:57:43.510736 31072 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c7f0e2a52f4482d9383ac26ebb6aea7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:57:43.602070 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:43.596043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57766 (local address 127.8.57.61:40067)
0929 02:57:43.596844 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:43.596849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:43.596866 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:57:43.596925 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:43.596932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:43.597091 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:43.597207 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:43.597679 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.598492 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.599810 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.600274 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.600382 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:43.600866 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:43.601401 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:43.601560 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:43.601694 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 02:57:43.601778 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":459,"threads_started":1}
I20260929 02:57:43.614110 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:43.607626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.8.57.60:34325)
0929 02:57:43.608114 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:43.608120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:43.608444 (+ 324us) server_negotiation.cc:407] Connection header received
0929 02:57:43.608644 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:43.608647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:43.608793 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:43.608922 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:43.609367 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.610117 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.612037 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.612420 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.612517 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:43.612992 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:43.613518 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:43.613663 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:43.613718 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 02:57:43.613781 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":248,"threads_started":1}
I20260929 02:57:43.635679 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:43.620115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.8.57.62:40829)
0929 02:57:43.627958 (+ 7843us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:43.627973 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:43.627995 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:57:43.628107 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:43.628112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:43.628330 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:43.628512 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:43.629089 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.630232 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.632238 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.632962 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.633088 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:43.633740 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:43.634182 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:43.634287 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:43.635284 (+ 997us) server_negotiation.cc:299] Negotiation successful
0929 02:57:43.635368 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7701,"thread_start_us":7627,"threads_started":1}
I20260929 02:57:43.636080 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:43.620272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.8.57.60:34325)
0929 02:57:43.620432 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:43.620437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:43.629153 (+ 8716us) server_negotiation.cc:407] Connection header received
0929 02:57:43.629404 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:43.629409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:43.629506 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:43.629565 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:43.630230 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.631276 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.633958 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.634595 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.634684 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:43.635305 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:43.635438 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:43.635498 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:43.635704 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:57:43.635785 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:57:43.641538 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:43.632551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57778 (local address 127.8.57.61:40067)
0929 02:57:43.632703 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:43.632708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:43.635209 (+ 2501us) server_negotiation.cc:407] Connection header received
0929 02:57:43.636175 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:43.636181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:43.636258 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:43.636315 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:43.636824 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.637874 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.639795 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:43.640434 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:43.640519 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:43.640953 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:43.641044 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:43.641119 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:43.641279 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 02:57:43.641344 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 02:57:43.645388 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30776
I20260929 02:57:43.646834 30838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:43.765558 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30776
W20260929 02:57:43.790271 30940 connection.cc:535] server connection from 127.0.0.1:43222 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 02:57:43.790639 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30850
I20260929 02:57:43.791808 30912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:43.903184 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30850
I20260929 02:57:43.918002 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 30924
I20260929 02:57:43.919024 30986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:44.023077 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 30924
2026-09-29T02:57:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4532 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:44 dist-test-slave-1dkt krb5kdc[31147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:44 dist-test-slave-1dkt krb5kdc[31147](info): set up 2 sockets
Sep 29 02:57:44 dist-test-slave-1dkt krb5kdc[31147](info): commencing operation
krb5kdc: starting...
W20260929 02:57:45.032522 8420 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/test-user.keytab.
Sep 29 02:57:45 dist-test-slave-1dkt krb5kdc[31147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.57.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
Entry for principal kudu/127.8.57.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab.
I20260929 02:57:45.269181 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36963
--webserver_interface=127.8.57.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:36963,127.8.57.61:34903,127.8.57.60:42113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:45.579401 31160 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:57:45.579725 31160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:45.579821 31160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:45.579953 31160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:45.587159 31160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:45.587253 31160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:45.587343 31160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:45.587406 31160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:45.587494 31160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:45.597072 31160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:36963,127.8.57.61:34903,127.8.57.60:42113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.62.keytab
--principal=kudu/127.8.57.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.62:36963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.57.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:45.598253 31160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:45.599517 31160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:45.610214 31168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:45.610491 31166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:45.610491 31165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:45.611286 31160 server_base.cc:1034] running on GCE node
I20260929 02:57:45.612038 31160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:45.613304 31160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:45.614514 31160 hybrid_clock.cc:648] HybridClock initialized: now 1790650665614441 us; error 46 us; skew 500 ppm
I20260929 02:57:45.614802 31160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:45 dist-test-slave-1dkt krb5kdc[31147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:45.622519 31160 init.cc:377] Logged in from keytab as kudu/127.8.57.62@KRBTEST.COM (short username kudu)
I20260929 02:57:45.623939 31160 webserver.cc:460] Webserver started at http://127.8.57.62:41341/ using document root <none> and password file <none>
I20260929 02:57:45.624601 31160 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:45.624693 31160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:45.625005 31160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:45.627861 31160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data/instance:
uuid: "f8963fd4e4ab48b0af733dad51c63b55"
format_stamp: "Formatted at 2026-09-29 02:57:45 on dist-test-slave-1dkt"
server_key: "b5fc9100fde4cf9cddd2a1b270e3be29"
server_key_iv: "53c590078b36e3cbaf4075199f7b8cb5"
server_key_version: "encryptionkey@0"
I20260929 02:57:45.628718 31160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal/instance:
uuid: "f8963fd4e4ab48b0af733dad51c63b55"
format_stamp: "Formatted at 2026-09-29 02:57:45 on dist-test-slave-1dkt"
server_key: "b5fc9100fde4cf9cddd2a1b270e3be29"
server_key_iv: "53c590078b36e3cbaf4075199f7b8cb5"
server_key_version: "encryptionkey@0"
I20260929 02:57:45.633513 31160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:57:45.636699 31175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:45.637872 31160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:57:45.638024 31160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal
uuid: "f8963fd4e4ab48b0af733dad51c63b55"
format_stamp: "Formatted at 2026-09-29 02:57:45 on dist-test-slave-1dkt"
server_key: "b5fc9100fde4cf9cddd2a1b270e3be29"
server_key_iv: "53c590078b36e3cbaf4075199f7b8cb5"
server_key_version: "encryptionkey@0"
I20260929 02:57:45.638206 31160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:45.649870 31160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:45.653437 31160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:45.653667 31160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:45.670265 31160 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.62:36963
I20260929 02:57:45.670284 31226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.62:36963 every 8 connection(s)
I20260929 02:57:45.671581 31160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:45.675197 31227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:45.679679 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 31160
I20260929 02:57:45.679849 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-0/wal/instance
I20260929 02:57:45.680291 8420 external_mini_cluster.cc:1428] Setting key 9fd6bb2ad7cee5b6f7f88b985ac99403
I20260929 02:57:45.681708 31227 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:57:45 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.61@KRBTEST.COM; defaulting to no policy
I20260929 02:57:45.702142 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.684755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36963 (local address 127.0.0.1:48806)
0929 02:57:45.685793 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.685826 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:45.687498 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:45.688299 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:45.688317 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:45.688964 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:45.689279 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:45.689295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:45.690755 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:45.690759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:45.691779 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:45.691790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:45.692633 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:45.692640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:45.692798 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:45.693380 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:45.693404 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:45.695375 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:45.698312 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:45.698322 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:45.698334 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:45.698687 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:45.699071 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:45.699076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:45.699079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:45.699218 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:45.699911 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:45.699918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:45.700137 (+ 219us) client_negotiation.cc:769] Sending connection context
0929 02:57:45.700343 (+ 206us) client_negotiation.cc:240] Negotiation successful
0929 02:57:45.700912 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":158,"thread_start_us":658,"threads_started":1}
I20260929 02:57:45.702124 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.684755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.8.57.62:36963)
0929 02:57:45.685516 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:45.685542 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:45.685933 (+ 391us) server_negotiation.cc:407] Connection header received
0929 02:57:45.687838 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:45.687844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:45.688103 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:45.688275 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:45.689472 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:45.690622 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:45.691925 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:45.692435 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:45.692600 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:45.695573 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:45.695580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:45.695591 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:45.695621 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:45.698205 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:45.698793 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:45.698799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:45.698801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:45.698964 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:45.699309 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:45.699314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:45.699317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:45.699762 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:45.699951 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:45.700473 (+ 522us) server_negotiation.cc:299] Negotiation successful
0929 02:57:45.700794 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":545,"thread_start_us":344,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.61@KRBTEST.COM" created.
I20260929 02:57:45.705042 31227 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:45.706005 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.705646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41158)
0929 02:57:45.705801 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.705900 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 02:57:45.706277 31176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
I20260929 02:57:45.707628 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.707303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41168)
0929 02:57:45.707433 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.707507 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 02:57:45.707806 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:34903: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab.
Entry for principal kudu/127.8.57.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab.
I20260929 02:57:45.730609 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:34903
--webserver_interface=127.8.57.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab
--principal=kudu/127.8.57.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:36963,127.8.57.61:34903,127.8.57.60:42113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:57:45.757102 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } attempt: 1
I20260929 02:57:45.758093 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.757692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41176)
0929 02:57:45.757869 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.757964 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 02:57:45.759025 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.758718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41184)
0929 02:57:45.758837 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.758902 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:57:45.759202 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:34903: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
I20260929 02:57:45.827405 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } attempt: 2
I20260929 02:57:45.828609 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.828138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41194)
0929 02:57:45.828372 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.828460 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 02:57:45.829595 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.829251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41202)
0929 02:57:45.829379 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.829455 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 02:57:45.829777 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:34903: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
I20260929 02:57:45.920992 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } attempt: 3
I20260929 02:57:45.922039 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.921618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41216)
0929 02:57:45.921808 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.921898 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 02:57:45.923050 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:45.922721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41224)
0929 02:57:45.922846 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:45.922921 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 02:57:45.923233 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:34903: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
W20260929 02:57:46.034147 31234 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:57:46.034441 31234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:46.034513 31234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:46.034592 31234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:46.041548 31234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:46.041646 31234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:46.041738 31234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:46.041810 31234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:46.041890 31234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:46.051291 31234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:36963,127.8.57.61:34903,127.8.57.60:42113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.61.keytab
--principal=kudu/127.8.57.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.61:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.57.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:46.052489 31234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:46.053756 31234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:46.064473 31239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:46.064445 31240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:46.064533 31242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:46.065081 31234 server_base.cc:1034] running on GCE node
I20260929 02:57:46.065691 31234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:46.066419 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } attempt: 4
I20260929 02:57:46.066936 31234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:46.067556 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.067004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41228)
0929 02:57:46.067304 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.067397 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260929 02:57:46.068178 31234 hybrid_clock.cc:648] HybridClock initialized: now 1790650666068064 us; error 79 us; skew 500 ppm
I20260929 02:57:46.068532 31234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:57:46.068548 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.068190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41244)
0929 02:57:46.068354 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.068435 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 02:57:46.068718 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.61:34903: Network error: Client connection negotiation failed: client connection to 127.8.57.61:34903: connect: Connection refused (error 111)
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:46.071403 31234 init.cc:377] Logged in from keytab as kudu/127.8.57.61@KRBTEST.COM (short username kudu)
I20260929 02:57:46.072746 31234 webserver.cc:460] Webserver started at http://127.8.57.61:46411/ using document root <none> and password file <none>
I20260929 02:57:46.073339 31234 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:46.073460 31234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:46.073741 31234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:46.076496 31234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data/instance:
uuid: "fe1be4441a4f472c8d4fd7cb77210a84"
format_stamp: "Formatted at 2026-09-29 02:57:46 on dist-test-slave-1dkt"
server_key: "86892c492fddf052faa04235ba4a376e"
server_key_iv: "1ef164db9d90532146b786534937b1b2"
server_key_version: "encryptionkey@0"
I20260929 02:57:46.077203 31234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal/instance:
uuid: "fe1be4441a4f472c8d4fd7cb77210a84"
format_stamp: "Formatted at 2026-09-29 02:57:46 on dist-test-slave-1dkt"
server_key: "86892c492fddf052faa04235ba4a376e"
server_key_iv: "1ef164db9d90532146b786534937b1b2"
server_key_version: "encryptionkey@0"
I20260929 02:57:46.081890 31234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:57:46.085013 31249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:46.086230 31234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:57:46.086397 31234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal
uuid: "fe1be4441a4f472c8d4fd7cb77210a84"
format_stamp: "Formatted at 2026-09-29 02:57:46 on dist-test-slave-1dkt"
server_key: "86892c492fddf052faa04235ba4a376e"
server_key_iv: "1ef164db9d90532146b786534937b1b2"
server_key_version: "encryptionkey@0"
I20260929 02:57:46.086583 31234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:46.113292 31234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:46.117054 31234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:46.117293 31234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:46.134407 31234 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.61:34903
I20260929 02:57:46.134405 31300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.61:34903 every 8 connection(s)
I20260929 02:57:46.135814 31234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/data/info.pb
I20260929 02:57:46.138887 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 31234
I20260929 02:57:46.139057 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-1/wal/instance
I20260929 02:57:46.139463 31301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:46.139482 8420 external_mini_cluster.cc:1428] Setting key aca3066305f7da78d08a681f90601d44
I20260929 02:57:46.146147 31301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.57.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.57.60@KRBTEST.COM" created.
I20260929 02:57:46.165143 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.148622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.8.57.62:36963)
0929 02:57:46.148776 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.148781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.150090 (+ 1309us) server_negotiation.cc:407] Connection header received
0929 02:57:46.151869 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.151899 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.151980 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.152072 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.153535 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.154217 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.155643 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.156076 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.156196 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.159727 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.159732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.159736 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.159768 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.162412 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.163058 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.163064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.163068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.163179 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.163558 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.163563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.163567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.163829 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.163972 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.164617 (+ 645us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.164825 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:57:46.166011 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.149221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36963 (local address 127.0.0.1:48810)
0929 02:57:46.149932 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.149974 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.151614 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.152295 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.152303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.152807 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.153376 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.153394 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.154360 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.154364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.155503 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.155516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.156289 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.156297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.156507 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.157078 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.157103 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.159525 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.162551 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.162560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.162572 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.162947 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.163286 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.163289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.163291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.163451 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.163969 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.163976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.164278 (+ 302us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.164514 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.164973 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":308,"threads_started":1}
I20260929 02:57:46.167474 31301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:46.181799 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.168218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41256)
0929 02:57:46.168412 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.168430 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.168528 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.169399 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.169403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.169841 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.170011 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.170022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.171827 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.171831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.172698 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.172709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.173661 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.173665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.173775 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.174249 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.174264 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.176214 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.179667 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.179672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.179675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.180055 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.180409 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.180412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.180414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.180541 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.181185 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.181192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.181293 (+ 101us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.181388 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.181562 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:57:46.182330 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.168388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41256 (local address 127.8.57.61:34903)
0929 02:57:46.169029 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.169034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.169058 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:57:46.169097 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.169103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.169256 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.169359 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.170171 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.171696 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.172822 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.173543 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.173646 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.176402 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.176413 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.176427 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.176463 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.179537 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.180171 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.180179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.180182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.180316 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.180649 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.180654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.180658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.181077 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.181284 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.181577 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.181821 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":356,"threads_started":1}
I20260929 02:57:46.184643 31301 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:46.185570 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.185176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49102)
0929 02:57:46.185323 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.185445 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 02:57:46.185786 31250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
I20260929 02:57:46.187121 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.186832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49106)
0929 02:57:46.186948 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.187022 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 02:57:46.187322 31301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.57.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab.
Entry for principal kudu/127.8.57.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab.
I20260929 02:57:46.191215 8420 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
/tmp/dist-test-taskV6jRaD/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:42113
--webserver_interface=127.8.57.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab
--principal=kudu/127.8.57.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.57.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.57.62:36963,127.8.57.61:34903,127.8.57.60:42113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:57:46.236579 31301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 1
I20260929 02:57:46.237690 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.237276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49118)
0929 02:57:46.237455 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.237542 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 02:57:46.238662 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.238340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49134)
0929 02:57:46.238480 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.238555 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:57:46.238878 31301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
I20260929 02:57:46.307086 31301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 2
I20260929 02:57:46.308230 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.307756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49140)
0929 02:57:46.307975 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.308076 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 02:57:46.309188 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.308871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49148)
0929 02:57:46.309004 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.309079 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 02:57:46.309386 31301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
I20260929 02:57:46.367951 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } attempt: 5
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:46.380580 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.368765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.8.57.61:34903)
0929 02:57:46.368956 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.368962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.368983 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:57:46.369104 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.369109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.369196 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.369275 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.370067 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.370756 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.371941 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.372388 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.372495 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.375602 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.375607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.375610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.375642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.378076 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.378859 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.378863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.378865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.378984 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.379355 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.379358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.379360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.379622 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.379727 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.380186 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.380351 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:57:46.380748 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.368645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41262)
0929 02:57:46.368826 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.368846 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.368970 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.369333 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.369338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.369750 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.369926 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.369935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.370919 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.370923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.371771 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.371790 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.372526 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.372532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.372735 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.373283 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.373301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.375405 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.378206 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.378211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.378215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.378746 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.379088 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.379093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.379097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.379265 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.379730 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.379734 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.379961 (+ 227us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.380112 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.380395 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 02:57:46.381762 31227 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:46.382736 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.382290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49158)
0929 02:57:46.382468 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.382572 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 02:57:46.383675 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.383315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49170)
0929 02:57:46.383451 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.383559 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 02:57:46.383911 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
I20260929 02:57:46.400561 31301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 3
I20260929 02:57:46.401645 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.401201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49180)
0929 02:57:46.401398 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.401487 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 02:57:46.402650 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.402258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49188)
0929 02:57:46.402463 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.402539 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260929 02:57:46.402840 31301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
I20260929 02:57:46.435123 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 1
I20260929 02:57:46.436256 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.435814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49198)
0929 02:57:46.436011 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.436100 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 02:57:46.437156 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.436879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49212)
0929 02:57:46.436991 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.437058 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 02:57:46.437361 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
W20260929 02:57:46.503243 31308 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:57:46.503557 31308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:46.503628 31308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:46.503707 31308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 02:57:46.505546 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 2
I20260929 02:57:46.506608 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.506167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49220)
0929 02:57:46.506357 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.506443 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 02:57:46.507458 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.507169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49222)
0929 02:57:46.507280 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.507343 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 02:57:46.507684 31227 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
W20260929 02:57:46.511361 31308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:46.511441 31308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:46.511528 31308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:46.511585 31308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:46.511664 31308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:46.521332 31308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.57.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.57.62:36963,127.8.57.61:34903,127.8.57.60:42113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/krb5kdc/kudu_127.8.57.60.keytab
--principal=kudu/127.8.57.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.57.60:42113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.57.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:46.522543 31308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:46.523805 31308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:46.534701 31313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:46.534781 31314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:46.534801 31316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:46.535365 31308 server_base.cc:1034] running on GCE node
I20260929 02:57:46.536087 31308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:46.537326 31308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:46.538555 31308 hybrid_clock.cc:648] HybridClock initialized: now 1790650666538470 us; error 40 us; skew 500 ppm
I20260929 02:57:46.538846 31308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:46.541803 31308 init.cc:377] Logged in from keytab as kudu/127.8.57.60@KRBTEST.COM (short username kudu)
I20260929 02:57:46.543179 31308 webserver.cc:460] Webserver started at http://127.8.57.60:37221/ using document root <none> and password file <none>
I20260929 02:57:46.543810 31308 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:46.543923 31308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:46.544209 31308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:46.546041 31301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 4
I20260929 02:57:46.547111 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.546591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49226)
0929 02:57:46.546815 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.546911 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 02:57:46.547230 31308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data/instance:
uuid: "db5b8977c9114114b0ba5855eeeb7f0e"
format_stamp: "Formatted at 2026-09-29 02:57:46 on dist-test-slave-1dkt"
server_key: "3c66f38ab01d5b308661b0f0cc79225f"
server_key_iv: "c7f53acdc272978e689df7bf22f7fe77"
server_key_version: "encryptionkey@0"
I20260929 02:57:46.548079 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.547687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49238)
0929 02:57:46.547835 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.547942 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:57:46.548081 31308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal/instance:
uuid: "db5b8977c9114114b0ba5855eeeb7f0e"
format_stamp: "Formatted at 2026-09-29 02:57:46 on dist-test-slave-1dkt"
server_key: "3c66f38ab01d5b308661b0f0cc79225f"
server_key_iv: "c7f53acdc272978e689df7bf22f7fe77"
server_key_version: "encryptionkey@0"
W20260929 02:57:46.548300 31301 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.57.60:42113: Network error: Client connection negotiation failed: client connection to 127.8.57.60:42113: connect: Connection refused (error 111)
I20260929 02:57:46.552769 31308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:57:46.555773 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:46.556970 31308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:57:46.557108 31308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data,/tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal
uuid: "db5b8977c9114114b0ba5855eeeb7f0e"
format_stamp: "Formatted at 2026-09-29 02:57:46 on dist-test-slave-1dkt"
server_key: "3c66f38ab01d5b308661b0f0cc79225f"
server_key_iv: "c7f53acdc272978e689df7bf22f7fe77"
server_key_version: "encryptionkey@0"
I20260929 02:57:46.557281 31308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:46.578938 31308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:46.582937 31308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:46.583169 31308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:46.599757 31308 rpc_server.cc:307] RPC server started. Bound to: 127.8.57.60:42113
I20260929 02:57:46.599777 31374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.57.60:42113 every 8 connection(s)
I20260929 02:57:46.600996 31308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/data/info.pb
I20260929 02:57:46.603232 8420 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu as pid 31308
I20260929 02:57:46.603384 8420 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskV6jRaD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650321430198-8420-0/minicluster-data/master-2/wal/instance
I20260929 02:57:46.603801 8420 external_mini_cluster.cc:1428] Setting key 164cd9a09a37711aac4b9adae6530875
I20260929 02:57:46.604408 31375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:46.612251 31375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:46.613883 31227 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 3
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.62@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
I20260929 02:57:46.626323 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.605372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.8.57.60:42113)
0929 02:57:46.606056 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.606068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.606105 (+ 37us) server_negotiation.cc:407] Connection header received
0929 02:57:46.607667 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.607687 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.608111 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.608468 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.609786 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.611069 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.612295 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.612956 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.613122 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.617628 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.617641 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.617654 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.617705 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.621379 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.622176 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.622185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.622193 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.622407 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.622828 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.622833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.622837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.623515 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.623780 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.624362 (+ 582us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.624991 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":291,"threads_started":1}
I20260929 02:57:46.630465 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.614468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49252)
0929 02:57:46.614757 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.614780 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.614883 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.616785 (+ 1902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.616791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.617196 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.617326 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.617334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.618890 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.618896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.620269 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.620284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.621202 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.621208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.621403 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.621920 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.621937 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.624552 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.627908 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.627912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.627915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.628341 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.628688 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.628693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.628696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.628834 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.629438 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.629442 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.629647 (+ 205us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.629767 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.630022 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260929 02:57:46.630864 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.614655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.8.57.62:36963)
0929 02:57:46.614835 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.614842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.615971 (+ 1129us) server_negotiation.cc:407] Connection header received
0929 02:57:46.616237 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.616242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.616314 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.616375 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.617905 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.618673 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.620664 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.621292 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.621374 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.625225 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.625232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.625236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.625272 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.627902 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.628554 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.628559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.628562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.628725 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.629162 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.629168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.629171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.629582 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.629714 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.630228 (+ 514us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.630448 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:57:46.631395 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.614596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.8.57.60:42113)
0929 02:57:46.616531 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.616534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.616553 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:57:46.616590 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.616593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.616643 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.616731 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.617484 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.618753 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.620433 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.621046 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.621165 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.624776 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.624783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.624787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.624827 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.627759 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.628451 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.628456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.628459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.628581 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.628954 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.628959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.628961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.629306 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.630809 (+ 1503us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.630903 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.631093 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1820,"thread_start_us":1708,"threads_started":1}
I20260929 02:57:46.632788 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.615065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.62:36963 (local address 127.0.0.1:48826)
0929 02:57:46.615830 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.615896 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.616095 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.616623 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.616630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.617416 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.617772 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.617782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.618834 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.618839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.620520 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.620531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.621645 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.621651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.621829 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.622413 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.622434 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.625049 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.628024 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.628033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.628039 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.628435 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.628826 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.628831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.628834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.629053 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.629704 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.629711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.629889 (+ 178us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.631439 (+ 1550us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.632192 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"spinlock_wait_cycles":23424,"thread_start_us":371,"threads_started":1}
I20260929 02:57:46.634027 31375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:46.637223 31227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55: Bootstrap starting.
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.62@KRBTEST.COM
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:46.640901 31227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:46.642102 31227 log.cc:824] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:46.645597 31227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55: No bootstrap required, opened a new log
I20260929 02:57:46.647090 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.634589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.61:34903 (local address 127.0.0.1:41268)
0929 02:57:46.634760 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.634776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.634853 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.635398 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.635402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.635816 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.636003 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.636011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.636908 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.636913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.637862 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.637874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.638538 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.638544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.638710 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.639147 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.639163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.641411 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.644698 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.644703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.644706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.645164 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.645560 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.645564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.645566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.645710 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.646268 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.646273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.646400 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.646512 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.646760 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 02:57:46.647594 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.634926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.8.57.61:34903)
0929 02:57:46.635110 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.635116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.635134 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:57:46.635187 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.635192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.635262 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.635351 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.636138 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.636784 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.637994 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.638402 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.638520 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.641619 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.641623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.641626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.641679 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.644588 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.645285 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.645290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.645293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.645452 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.645802 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.645807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.645811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.646153 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.646292 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.646622 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.646825 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:57:46.648398 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.630107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.8.57.62:36963)
0929 02:57:46.631087 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.631093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.631110 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:57:46.631158 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.631163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.631233 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.631328 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.633094 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.634371 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.635805 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.636405 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.636525 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.641370 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.641375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.641380 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.641411 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.644504 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.645596 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.645600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.645603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.645727 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.646128 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.646137 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.646140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.646485 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.647665 (+ 1180us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.647783 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.648024 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":3184,"threads_started":1}
I20260929 02:57:46.648504 31375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:57:46.655588 31227 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.656013 31227 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8963fd4e4ab48b0af733dad51c63b55, State: Initialized, Role: FOLLOWER
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.60@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
I20260929 02:57:46.656541 31227 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.657440 31383 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.657912 31383 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: This master's current role is: FOLLOWER
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.57.61@KRBTEST.COM
I20260929 02:57:46.658735 31227 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:46.662572 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.649115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49268)
0929 02:57:46.649371 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.649387 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.649477 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.649900 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.649904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.650330 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.650493 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.650504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.651783 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.651788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.653115 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.653128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.654119 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.654125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.654271 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.655035 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.655058 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.656968 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.659839 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.659843 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.659847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.660384 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.660720 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.660724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.660728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.660846 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.661474 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.661481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.661649 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.661803 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.662130 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260929 02:57:46.662612 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.649375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.8.57.60:42113)
0929 02:57:46.649537 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.649543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.649560 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:57:46.649603 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.649608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.649685 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.649771 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.650625 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.651641 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.653257 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.653859 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.653966 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.657148 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.657155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.657159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.657190 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.659707 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.660481 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.660504 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.660507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.660626 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.660943 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.660948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.660952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.661260 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.661472 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.661952 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.662282 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":58}
I20260929 02:57:46.665701 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.650253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.8.57.61:34903)
0929 02:57:46.650392 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.650397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.650413 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:57:46.650521 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.650526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.650583 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.650655 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.651739 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.652447 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.653746 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.654373 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.654531 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.659381 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.659387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.659391 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.659436 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.662898 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.663686 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.663690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.663693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.663831 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.664297 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.664302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.664304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.664634 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.664881 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.665163 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.665374 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:57:46.668179 31375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: Bootstrap starting.
I20260929 02:57:46.670629 8420 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:57:46.672163 31375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:46.673426 31375 log.cc:824] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:46.673516 31202 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f8963fd4e4ab48b0af733dad51c63b55"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48830
I20260929 02:57:46.673877 31202 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 02:57:46.674068 31202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:46.677736 31202 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.679654 31375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: No bootstrap required, opened a new log
I20260929 02:57:46.680873 31202 leader_election.cc:288] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [CANDIDATE]: Term 1 election: Requested vote from peers fe1be4441a4f472c8d4fd7cb77210a84 (127.8.57.61:34903), db5b8977c9114114b0ba5855eeeb7f0e (127.8.57.60:42113)
I20260929 02:57:46.690867 31375 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.691336 31375 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5b8977c9114114b0ba5855eeeb7f0e, State: Initialized, Role: FOLLOWER
I20260929 02:57:46.691918 31375 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.693066 31395 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.693742 31395 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:46.694324 31375 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:46.694347 31350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8963fd4e4ab48b0af733dad51c63b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "db5b8977c9114114b0ba5855eeeb7f0e"
I20260929 02:57:46.694859 31350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:46.698508 31350 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8963fd4e4ab48b0af733dad51c63b55 in term 1.
I20260929 02:57:46.699304 31176 leader_election.cc:302] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db5b8977c9114114b0ba5855eeeb7f0e, f8963fd4e4ab48b0af733dad51c63b55; no voters:
I20260929 02:57:46.699819 31383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:46.700470 31383 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [term 1 LEADER]: Becoming Leader. State: Replica: f8963fd4e4ab48b0af733dad51c63b55, State: Running, Role: LEADER
I20260929 02:57:46.701159 31383 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.704902 31401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8963fd4e4ab48b0af733dad51c63b55. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.705334 31401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:46.706139 31403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:46.722257 31403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:46.710805 31160 master_runner.cc:186] Error getting master registration for 127.8.57.61:34903: OK, instance_id { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" instance_seqno: 1790650666129224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:57:46.725291 31160 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:57:46.744757 31350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [term 1 FOLLOWER]: Refusing update from remote peer f8963fd4e4ab48b0af733dad51c63b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:46.746383 31401 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:46.756522 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.721001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41284 (local address 127.8.57.61:34903)
0929 02:57:46.721192 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.721199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.726202 (+ 5003us) server_negotiation.cc:407] Connection header received
0929 02:57:46.726556 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.726562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.726640 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.726708 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.730303 (+ 3595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.731466 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.734252 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.734834 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.734931 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.740198 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.740204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.740209 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.740250 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.743784 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.753349 (+ 9565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.753356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.753360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.753538 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.754089 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.754095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.754098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.754459 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.754716 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.755917 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.756171 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:57:46.776453 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.727658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.8.57.62:36963)
0929 02:57:46.727824 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.727830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.744621 (+ 16791us) server_negotiation.cc:407] Connection header received
0929 02:57:46.744743 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.744749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.744833 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.744931 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.755610 (+ 10679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.756835 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.761846 (+ 5011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.762594 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.762700 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.766417 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.766425 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.766431 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.766474 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.770752 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.772095 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.772104 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.772109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.772430 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.773178 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.773184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.773214 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.773880 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.774318 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.775276 (+ 958us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.775735 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:57:46.787927 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.726298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.8.57.60:42113)
0929 02:57:46.726476 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.726483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.740421 (+ 13938us) server_negotiation.cc:407] Connection header received
0929 02:57:46.740667 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.740673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.740783 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.740857 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.744781 (+ 3924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.745750 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.751458 (+ 5708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.752311 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.752414 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.768978 (+ 16564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.768986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.768992 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.769038 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.783182 (+ 14144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.784232 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.784238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.784242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.784422 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.785259 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.785265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.785269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.785654 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.787089 (+ 1435us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.787242 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.787479 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:57:46.805145 31395 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8963fd4e4ab48b0af733dad51c63b55. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.805452 31395 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:46.807848 31413 mvcc.cc:205] Tried to move back new op lower bound from 7334505130940780544 to 7334505130813390848. Current Snapshot: MvccSnapshot[applied={T|T < 7334505130940780544}]
I20260929 02:57:46.809453 31418 mvcc.cc:205] Tried to move back new op lower bound from 7334505130940780544 to 7334505130813390848. Current Snapshot: MvccSnapshot[applied={T|T < 7334505130940780544}]
I20260929 02:57:46.810182 31395 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.810422 31395 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:46.819276 31403 catalog_manager.cc:1347] Generated new cluster ID: 42b3cd352def4f3aba54de3d2152fc38
I20260929 02:57:46.819383 31403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260929 02:57:46.819623 31421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:57:46.819947 31421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:57:46.821249 31383 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.821560 31383 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:46.824072 31308 master_runner.cc:186] Error getting master registration for 127.8.57.61:34903: OK, instance_id { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" instance_seqno: 1790650666129224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:57:46.836550 31308 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:57:46.847499 31301 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } attempt: 5
I20260929 02:57:46.850258 31403 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:46.852480 31403 catalog_manager.cc:1504] Loading token signing keys...
Sep 29 02:57:46 dist-test-slave-1dkt krb5kdc[31147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.57.61@KRBTEST.COM for kudu/127.8.57.60@KRBTEST.COM
I20260929 02:57:46.866569 31403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55: Generated new TSK 0
I20260929 02:57:46.867579 31403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:57:46.871069 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.848199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.57.60:42113 (local address 127.0.0.1:49292)
0929 02:57:46.848396 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:46.848418 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:46.848530 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:46.849314 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:46.849319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:46.849850 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:46.850068 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.850081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.851453 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.851459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.852823 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:46.852840 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.853979 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.853986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:46.854205 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.854980 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:46.855003 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:46.858494 (+ 3491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:46.867737 (+ 9243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.867743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.867747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.868392 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.868858 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:46.868862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:46.868866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:46.869116 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:46.870028 (+ 912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:46.870058 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:46.870263 (+ 205us) client_negotiation.cc:769] Sending connection context
0929 02:57:46.870401 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 02:57:46.870706 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 02:57:46.871646 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:46.848720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.8.57.60:42113)
0929 02:57:46.848885 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:46.848891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:46.848908 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:57:46.848957 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:46.848961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:46.849029 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:46.849132 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:46.850236 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.851292 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.852996 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:46.853670 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:46.853786 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:46.858694 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:46.858701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:46.858706 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:46.858743 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:46.867546 (+ 8803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.868537 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.868542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.868546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.868701 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:46.869277 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:46.869283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:46.869287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:46.869900 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:46.870056 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:46.870636 (+ 580us) server_negotiation.cc:299] Negotiation successful
0929 02:57:46.871049 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:57:46.877542 31301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: Bootstrap starting.
I20260929 02:57:46.882967 31301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:46.884358 31301 log.cc:824] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:46.907047 31301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: No bootstrap required, opened a new log
I20260929 02:57:46.929814 31301 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.930303 31301 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe1be4441a4f472c8d4fd7cb77210a84, State: Initialized, Role: FOLLOWER
I20260929 02:57:46.930853 31301 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } }
I20260929 02:57:46.932499 31428 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:46.933748 31428 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:46.938608 31301 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:46.939028 31275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:46.938632 31276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8963fd4e4ab48b0af733dad51c63b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fe1be4441a4f472c8d4fd7cb77210a84"
I20260929 02:57:46.944234 31275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [term 1 FOLLOWER]: Refusing update from remote peer f8963fd4e4ab48b0af733dad51c63b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:57:46.947971 31276 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8963fd4e4ab48b0af733dad51c63b55 in term 1.
I20260929 02:57:46.950615 31401 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:57:47.004879 31438 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:47.005303 31438 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:57:47.005607 31428 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8963fd4e4ab48b0af733dad51c63b55. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:47.005785 31428 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 02:57:47.026422 31305 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.016941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41286 (local address 127.8.57.61:34903)
0929 02:57:47.017115 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.017120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.024490 (+ 7370us) server_negotiation.cc:407] Connection header received
0929 02:57:47.024750 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.024756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.024855 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.024926 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.026123 (+ 1197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41286: BlockingRecv error: recv got EOF from 127.0.0.1:41286 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 02:57:47.028777 31229 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.016611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.8.57.62:36963)
0929 02:57:47.016797 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.016803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.027163 (+ 10360us) server_negotiation.cc:407] Connection header received
0929 02:57:47.027402 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.027408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.027497 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.027564 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.028545 (+ 981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48854: BlockingRecv error: recv got EOF from 127.0.0.1:48854 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 02:57:47.034490 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.032259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.8.57.60:42113)
0929 02:57:47.032426 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.032431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.032962 (+ 531us) server_negotiation.cc:407] Connection header received
0929 02:57:47.033170 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.033175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.033251 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.033315 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.034250 (+ 935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49300: BlockingRecv error: recv got EOF from 127.0.0.1:49300 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:57:47.037737 31448 catalog_manager.cc:1259] Loaded cluster ID: 42b3cd352def4f3aba54de3d2152fc38
I20260929 02:57:47.037837 31448 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: loading cluster ID for follower catalog manager: success
I20260929 02:57:47.041580 31448 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: acquiring CA information for follower catalog manager: success
I20260929 02:57:47.044200 31448 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe1be4441a4f472c8d4fd7cb77210a84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:57:47.092603 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.066676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.8.57.61:34903)
0929 02:57:47.066862 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.066869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.084420 (+ 17551us) server_negotiation.cc:407] Connection header received
0929 02:57:47.084653 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.084659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.084881 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.085032 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.085704 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.086859 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.089195 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.089885 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.089980 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.090741 (+ 761us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.091397 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.091524 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.092159 (+ 635us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.092259 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 02:57:47.149880 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.125475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.8.57.60:42113)
0929 02:57:47.125664 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.125670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.138417 (+ 12747us) server_negotiation.cc:407] Connection header received
0929 02:57:47.148312 (+ 9895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.148340 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.148492 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.148584 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.149647 (+ 1063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49316: BlockingRecv error: recv got EOF from 127.0.0.1:49316 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:57:47.219053 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.205985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.8.57.61:34903)
0929 02:57:47.206214 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.206220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.212416 (+ 6196us) server_negotiation.cc:407] Connection header received
0929 02:57:47.212625 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.212632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.212747 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.212819 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.213597 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.214719 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.216751 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.217461 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.217565 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.218190 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.218352 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.218414 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.218635 (+ 221us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.218724 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260929 02:57:47.282222 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.268543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.8.57.60:42113)
0929 02:57:47.268729 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.268735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.280459 (+ 11724us) server_negotiation.cc:407] Connection header received
0929 02:57:47.280671 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.280677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.280779 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.280917 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.281991 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49320: BlockingRecv error: recv got EOF from 127.0.0.1:49320 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:57:47.333300 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.321941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.8.57.61:34903)
0929 02:57:47.322126 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.322132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.327204 (+ 5072us) server_negotiation.cc:407] Connection header received
0929 02:57:47.327390 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.327396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.327497 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.327615 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.328144 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.329313 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.331198 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.331920 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.332045 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.332485 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.332623 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.332715 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.332875 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.332967 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:57:47.341872 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.339755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.8.57.60:42113)
0929 02:57:47.339978 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.339984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.340480 (+ 496us) server_negotiation.cc:407] Connection header received
0929 02:57:47.340657 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.340663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.340754 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.340845 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.341630 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49332: BlockingRecv error: recv got EOF from 127.0.0.1:49332 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:57:47.355846 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.349236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.8.57.61:34903)
0929 02:57:47.349453 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.349459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.349913 (+ 454us) server_negotiation.cc:407] Connection header received
0929 02:57:47.350096 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.350102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.350182 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.350269 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.350710 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.351826 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.353697 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.354429 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.354548 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.355033 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.355193 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.355286 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.355503 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.355589 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
W20260929 02:57:47.378050 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.371830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.8.57.60:42113)
0929 02:57:47.372105 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.372111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.376467 (+ 4356us) server_negotiation.cc:407] Connection header received
0929 02:57:47.376675 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.376682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.376781 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.376902 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.377782 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49346: BlockingRecv error: recv got EOF from 127.0.0.1:49346 (error 108)
Metrics: {"server-negotiator.queue_time_us":143}
I20260929 02:57:47.396378 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.389403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.8.57.61:34903)
0929 02:57:47.389608 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.389614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.390126 (+ 512us) server_negotiation.cc:407] Connection header received
0929 02:57:47.390441 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.390448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.390545 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.390681 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.391145 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.392338 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.394218 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.394893 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.395037 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.395494 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.395636 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.395738 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.395919 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.396021 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:57:47.407948 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.405769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.8.57.60:42113)
0929 02:57:47.405957 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.405963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.406486 (+ 523us) server_negotiation.cc:407] Connection header received
0929 02:57:47.406664 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.406670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.406765 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.406881 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.407693 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49362: BlockingRecv error: recv got EOF from 127.0.0.1:49362 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:57:47.434017 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.428138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.8.57.61:34903)
0929 02:57:47.428359 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.428365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.428674 (+ 309us) server_negotiation.cc:407] Connection header received
0929 02:57:47.428832 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.428838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.428935 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.429067 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.429503 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.430747 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.432114 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.432765 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.432887 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.433303 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.433436 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.433536 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.433652 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.433735 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260929 02:57:47.441294 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.439417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.8.57.60:42113)
0929 02:57:47.439645 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.439651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.439910 (+ 259us) server_negotiation.cc:407] Connection header received
0929 02:57:47.440078 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.440084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.440214 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.440331 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.441056 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49364: BlockingRecv error: recv got EOF from 127.0.0.1:49364 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 02:57:47.482648 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.476423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.8.57.61:34903)
0929 02:57:47.476699 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.476705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.477057 (+ 352us) server_negotiation.cc:407] Connection header received
0929 02:57:47.477287 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.477293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.477375 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.477471 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.477912 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.478676 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.480956 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.481403 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.481497 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.481964 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.482086 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.482153 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.482288 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.482357 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260929 02:57:47.490177 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.488186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.8.57.60:42113)
0929 02:57:47.488415 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.488420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.488775 (+ 355us) server_negotiation.cc:407] Connection header received
0929 02:57:47.488941 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.488945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.489013 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.489097 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.489976 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49374: BlockingRecv error: recv got EOF from 127.0.0.1:49374 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:57:47.562531 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.557691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.8.57.61:34903)
0929 02:57:47.557918 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.557925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.558142 (+ 217us) server_negotiation.cc:407] Connection header received
0929 02:57:47.558337 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.558344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.558417 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.558493 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.558924 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.559665 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.561001 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.561387 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.561497 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.561888 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.561988 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.562060 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.562201 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.562278 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 02:57:47.569324 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.567519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.8.57.60:42113)
0929 02:57:47.567730 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.567736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.568019 (+ 283us) server_negotiation.cc:407] Connection header received
0929 02:57:47.568183 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.568189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.568262 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.568356 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.569112 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49390: BlockingRecv error: recv got EOF from 127.0.0.1:49390 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:57:47.570891 31401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8963fd4e4ab48b0af733dad51c63b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8963fd4e4ab48b0af733dad51c63b55" member_type: VOTER last_known_addr { host: "127.8.57.62" port: 36963 } } peers { permanent_uuid: "fe1be4441a4f472c8d4fd7cb77210a84" member_type: VOTER last_known_addr { host: "127.8.57.61" port: 34903 } } peers { permanent_uuid: "db5b8977c9114114b0ba5855eeeb7f0e" member_type: VOTER last_known_addr { host: "127.8.57.60" port: 42113 } } }
I20260929 02:57:47.571156 31401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8963fd4e4ab48b0af733dad51c63b55 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:47.705897 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.700697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.8.57.61:34903)
0929 02:57:47.700885 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.700890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.701285 (+ 395us) server_negotiation.cc:407] Connection header received
0929 02:57:47.701510 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.701515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.701590 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.701676 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.702138 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.702860 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.704269 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.704692 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.704815 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.705209 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.705316 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.705390 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.705535 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.705618 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 02:57:47.713061 31379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:57:47.711240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.8.57.60:42113)
0929 02:57:47.711478 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.711484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.711712 (+ 228us) server_negotiation.cc:407] Connection header received
0929 02:57:47.711848 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.711852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.711954 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.712035 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:47.712855 (+ 820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49404: BlockingRecv error: recv got EOF from 127.0.0.1:49404 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:57:47.821738 31421 catalog_manager.cc:1259] Loaded cluster ID: 42b3cd352def4f3aba54de3d2152fc38
I20260929 02:57:47.821841 31421 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: loading cluster ID for follower catalog manager: success
I20260929 02:57:47.824278 31421 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: acquiring CA information for follower catalog manager: success
I20260929 02:57:47.825980 31421 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db5b8977c9114114b0ba5855eeeb7f0e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:57:47.978168 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.973472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.8.57.61:34903)
0929 02:57:47.973719 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.973725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.973871 (+ 146us) server_negotiation.cc:407] Connection header received
0929 02:57:47.974070 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.974075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.974142 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.974218 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.974632 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.975330 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.976651 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.977039 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.977148 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.977526 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.977635 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.977707 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.977853 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.977929 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 02:57:47.988674 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.983281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.8.57.60:42113)
0929 02:57:47.983478 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.983484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.983595 (+ 111us) server_negotiation.cc:407] Connection header received
0929 02:57:47.983786 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.983791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.983983 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.984098 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.984492 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.985197 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.986490 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.986878 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.986984 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:47.987453 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:47.988096 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:47.988233 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:47.988315 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 02:57:47.988407 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:57:48.000819 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.994017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44106 (local address 127.8.57.61:34903)
0929 02:57:47.994178 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.994184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.995070 (+ 886us) server_negotiation.cc:407] Connection header received
0929 02:57:47.995303 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.995308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.995390 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.995454 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.996156 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.997055 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.999005 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.999531 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.999636 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:48.000031 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:48.000170 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:48.000261 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:48.000448 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:57:48.000526 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:57:48.001734 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.995111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52120 (local address 127.8.57.60:42113)
0929 02:57:47.995284 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.995290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.995904 (+ 614us) server_negotiation.cc:407] Connection header received
0929 02:57:47.996075 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.996079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.996155 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.996246 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.996817 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.997745 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.999333 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.999960 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:48.000079 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:48.001075 (+ 996us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:48.001186 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:48.001278 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:48.001403 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 02:57:48.001476 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:57:48.002314 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:47.994009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.8.57.62:36963)
0929 02:57:47.994408 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:47.994414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:47.995022 (+ 608us) server_negotiation.cc:407] Connection header received
0929 02:57:47.995368 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:47.995374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:47.995587 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:47.995720 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:57:47.996414 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.997318 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:47.999301 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:47.999960 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:48.000080 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:48.000777 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:57:48.001231 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:57:48.001330 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:48.001980 (+ 650us) server_negotiation.cc:299] Negotiation successful
0929 02:57:48.002065 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":189,"threads_started":1}
I20260929 02:57:48.006599 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 31160
I20260929 02:57:48.007494 31222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:48.115044 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 31160
I20260929 02:57:48.131690 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 31234
I20260929 02:57:48.132720 31296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:48.240029 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 31234
I20260929 02:57:48.255713 8420 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu pid 31308
I20260929 02:57:48.256711 31370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:48.363364 8420 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskV6jRaD/build/asan/bin/kudu with pid 31308
2026-09-29T02:57:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4336 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8868 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (346896 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST